Semantic segmentation method, device and equipment based on star chain 4D radar and medium

Through the semantic segmentation method based on Starlink 4D radar and combined with dual super resolution learning processing, the problem of insufficient semantic segmentation accuracy in the existing technology is solved, efficient semantic segmentation on resource-constrained devices is realized, and surgical image processing is improved accuracy and efficiency.

CN119090897APending Publication Date: 2024-12-06XIAMEN TANHONG INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411144045.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing XR surgical image processing technology has insufficient accuracy in semantic segmentation, especially in resource-constrained devices, which affects the accuracy and efficiency of the surgery.

Method used

Using the semantic segmentation method based on Starlink 4D radar, the interconnected data sets of communication satellites and 4D radar components are constructed, internal reference, external parameters and time calibration is performed, combined with dual super resolution learning processing, and semantic segmentation super resolution modules and single Image super-resolution module and feature similarity module obtain high resolution and surgical area images containing detailed semantic information.

Benefits of technology

Without adding additional computing costs, the accuracy of semantic segmentation is improved, making this technology more suitable for resource-constrained devices and improving the accuracy and efficiency of surgical image processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090897A_ABST
    Figure CN119090897A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a semantic segmentation method and device based on a star chain 4D radar, equipment and a medium. The method comprises the following steps: constructing a communication satellite and 4D radar component interconnection data set; internal reference calibration, external reference calibration and time calibration are carried out on a 4D imaging radar and a laser radar of the 4D radar assembly; aligning the data by using the arrival time as a timestamp, and projecting the 4D radar and laser radar point clouds into an image to obtain a fused image; and performing double super-resolution learning processing on the fused image to obtain an operation area image which has high resolution and contains detailed semantic information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a semantic segmentation method, device, equipment and medium based on Starlink 4D radar. Background Art

[0002] Existing technologies for XR surgical image processing mainly include a series of advanced image processing and computer vision technologies, which provide visual support for surgery with high precision, high definition and real-time feedback.

[0003] The following is a detailed overview of the current state of the art in image processing for XR surgery:

[0004] 1. Image acquisition and preprocessing

[0005] Multimodal image fusion: XR surgical systems usually combine multiple image modalities, such as CT, MRI, ultrasound, etc. Through multimodal image fusion technology, image information from different sources is integrated together to provide a more comprehensive surgical field of view.

[0006] The fusion algorithm can handle the registration, fusion and enhancement between images of different modalities, ensuring that the surgeon can accurately understand the structure of the lesion and its surrounding tissues.

[0007] Image denoising and enhancement: Surgical images may contain noise and artifacts, which may affect image quality. Image denoising techniques, such as filtering and wavelet transform, can effectively remove noise and improve image signal-to-noise ratio.

[0008] Image enhancement technology is used to improve the visual effects of images, such as contrast enhancement and sharpening, making lesions and surrounding tissues more clearly identifiable.

[0009] 2. 3D Reconstruction and Visualization

[0010] 3D reconstruction technology: 3D reconstruction algorithms are used to convert 2D surgical images into 3D models. These models can show the spatial structure and anatomical relationship of the lesion and its surrounding tissues, providing surgeons with an intuitive surgical field of view.

[0011] Three-dimensional reconstruction technology includes voxel reconstruction, surface reconstruction and other methods. The appropriate reconstruction algorithm can be selected according to surgical requirements.

[0012] Real-time visualization: XR surgical systems usually have real-time visualization capabilities, which can display the three-dimensional model of the lesion and its surrounding tissues in real time during the operation. This helps surgeons adjust the surgical plan at any time during the operation to ensure the accuracy and safety of the operation.

[0013] 3. Image Segmentation and Recognition

[0014] Image segmentation: Image segmentation technology is used to segment lesions, blood vessels, tissues, etc. in surgical images to form independent image areas. This helps surgeons to more accurately understand information such as the location, shape, and size of the lesions.

[0015] Commonly used image segmentation methods include threshold segmentation, region growing, level set and so on.

[0016] Target recognition and tracking: The XR surgical system also has target recognition and tracking functions, which can automatically identify and track key targets during surgery (such as lesions, surgical instruments, etc.). This helps surgeons keep their focus on key targets during surgery and improves the accuracy and efficiency of surgery.

[0017] 4. Advanced Image Processing Technology

[0018] Deep learning: In recent years, deep learning technology has been widely used in XR surgical image processing. By training deep learning models, tasks such as automatic segmentation, recognition, and classification of surgical images can be achieved.

[0019] The deep learning model has powerful feature extraction and learning capabilities, can process complex surgical image data, and improve the accuracy and efficiency of image processing.

[0020] Virtual reality and augmented reality: XR surgical systems also combine virtual reality (VR) and augmented reality (AR) technologies to provide surgeons with an immersive surgical experience. By wearing devices such as VR helmets or AR glasses, surgeons can perform surgical simulations, planning, and navigation in a virtual environment.

[0021] Virtual reality technology can simulate real surgical scenes and operating procedures to help surgeons become familiar with the surgical process and master surgical skills; augmented reality technology can overlay virtual information (such as surgical navigation lines, lesion markers, etc.) in real time during the operation to provide surgeons with real-time surgical guidance.

[0022] In summary, the existing technologies for XR surgical image processing cover multiple aspects, including image acquisition and preprocessing, 3D reconstruction and visualization, image segmentation and recognition, and advanced image processing technology. The application of these technologies provides surgeons with high-precision, high-definition, and real-time feedback visual support, which helps improve the accuracy, safety, and efficiency of surgery. Summary of the invention

[0023] Based on this, it is necessary to propose a semantic segmentation method, device, equipment and medium based on Starlink 4D radar to address the above problems.

[0024] A semantic segmentation method based on Starlink 4D radar, the method comprising:

[0025] Build a data set for interconnection between communication satellites and 4D radar components;

[0026] Performing internal parameter calibration, external parameter calibration and time calibration on the 4D imaging radar and the laser radar of the 4D radar assembly;

[0027] The arrival time is used as the timestamp to align the data, and the 4D radar and lidar point clouds are projected into the image to obtain a fused image;

[0028] The fused image is subjected to dual super-resolution learning processing to obtain an image of the surgical area having both high resolution and detailed semantic information.

[0029] In the above scheme, the communication satellite and 4D radar component interconnection data set is constructed, including a continuous sequence of 4D radar point clouds and 3D annotation content.

[0030] The above solution also includes 4D radar components, XR surgical robots, and satellites to form mission-critical IoT slices.

[0031] In the above scheme, in the interconnected mission-critical IoT slice, the IP / MPLS-SDN slice network connects the edge cloud and VMs in the space computing through SDN; the space computing has virtualized servers, and the server's Hypervisor runs a built-in vRouter / vSwitch. The SDN controller is responsible for creating SDN tunnels between the virtual servers and the DCG / W routers. Subsequently, the SDN controller performs mapping between the SDN tunnels and the MPLS L3 VPN to establish a connection between the space computing and the edge cloud.

[0032] The interconnected mission-critical IoT slices are formed through Beidou satellite timestamp slices, which transmit periodic data and non-periodic data in slices in units of time slots, and achieve synchronous transmission of data through a time-sensitive network.

[0033] In the above scheme, the dual super-resolution learning includes a semantic segmentation super-resolution module (SSSR), a single image super-resolution module (SISR), and a feature similarity module (FA).

[0034] In the above scheme, the input image is enlarged twice and then output through the semantic segmentation super-resolution module; and a high-resolution image is constructed from the low-resolution input through the single image super-resolution module.

[0035] In the above scheme, the feature similarity module guides SSSR to learn high-resolution representation

[0036]

[0037] The feature similarity module learns the matrix similarity between SISR and SSSR. The matrix similarity mainly describes the pairwise relationship between pixels. Among them, W'×H' represents the spatial size, we formulate the relationship between every two pixels, so the relationship graph contains W'H'×W'H' connections, and Sij represents the relationship between the i-th pixel and the j-th pixel on the feature map F.

[0038] A semantic segmentation device based on Starlink 4D radar, the device comprising: a data set unit, a calibration unit, and an image processing unit;

[0039] The data set unit is used to construct a communication satellite and 4D radar component interconnection data set;

[0040] The calibration unit is used to perform internal parameter calibration, external parameter calibration and time calibration on the 4D imaging radar and the laser radar of the 4D radar assembly;

[0041] The image processing unit is used to align data using the arrival time as a timestamp, and the 4D radar and lidar point clouds are projected into the image to obtain a fused image; it is also used to perform dual super-resolution learning processing on the fused image to obtain a surgical area image that has both high resolution and detailed semantic information.

[0042] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0043] Build a data set for interconnection between communication satellites and 4D radar components;

[0044] Performing internal parameter calibration, external parameter calibration and time calibration on the 4D imaging radar and the laser radar of the 4D radar assembly;

[0045] The arrival time is used as the timestamp to align the data, and the 4D radar and lidar point clouds are projected into the image to obtain a fused image;

[0046] The fused image is subjected to dual super-resolution learning processing to obtain an image of the surgical area having both high resolution and detailed semantic information.

[0047] A computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the following steps:

[0048] Build a data set for interconnection between communication satellites and 4D radar components;

[0049] Performing internal parameter calibration, external parameter calibration and time calibration on the 4D imaging radar and the laser radar of the 4D radar assembly;

[0050] The arrival time is used as the timestamp to align the data, and the 4D radar and lidar point clouds are projected into the image to obtain a fused image;

[0051] The fused image is subjected to dual super-resolution learning processing to obtain an image of the surgical area having both high resolution and detailed semantic information.

[0052] The embodiments of the present invention have the following beneficial effects:

[0053] The dual super-resolution learning (DSRL) framework proposed in the present invention can improve the accuracy of semantic segmentation without increasing additional computational cost, making the technology more suitable for resource-constrained devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] in:

[0056] Figure 1 Flow chart of a semantic segmentation method based on Starlink 4D radar in one embodiment;

[0057] Figure 2 A framework diagram of dual super-resolution learning in a semantic segmentation method based on Starlink 4D radar in one embodiment;

[0058] Figure 3 It is a visualization diagram of SSSR and SISR of the same input (0.5 times) in a semantic segmentation method based on Starlink 4D radar in one embodiment;

[0059] Figure 4 Schematic diagram of processing SSSR and SISR of the same input (0.5 times) in a semantic segmentation method based on Starlink 4D radar in an embodiment. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] In one embodiment, a semantic segmentation method based on Starlink 4D radar is provided, such as Figure 1 As shown, the method includes:

[0062] Step 101: construct a communication satellite and 4D radar component interconnection data set;

[0063] Specifically, the dataset contains a continuous sequence of 4D radar point clouds and 3D annotation content, and also provides multi-modal complete information on the interaction between lidar, camera, global satellite navigation system and communication satellite system.

[0064] The dataset contains 40K frames of synchronized data, of which 7757 frames and 44 sequences have high-quality annotated 3D bounding boxes and track IDs.

[0065] This dataset is suitable for developing 3D perception algorithms based on 4D radar to promote its application in advanced surgical imaging transmission. It provides a baseline for implementing 3D target detection based on 4D radar based on the dataset, and the results show that 4D imaging radar has broad application prospects in high-level surgical image transmission.

[0066] Step 102: performing internal parameter calibration, external parameter calibration and time calibration on the 4D imaging radar and the laser radar of the 4D radar assembly;

[0067] Specifically, the 4D radar assembly includes a 4D imaging radar, a lidar and a camera. The camera and lidar are installed on a bracket outside the operating room, and the 4D imaging radar is installed in the middle of the bracket outside the operating room. The lidar scans environmental information 360 degrees, while the camera and 4D radar can capture information in the front field of view (FOV), covering the front surgical transmission field of view.

[0068] In addition, GNSS information is included and corrected by real-time kinematics (RTK) to achieve high-precision positioning, which includes the speed and position information transmitted by the self-operation. Multi-sensor calibration is the basis of the perception algorithm, and the process mainly includes internal parameter calibration, external parameter calibration and time calibration.

[0069] The intrinsic parameters and distortion coefficients of the camera are calibrated using the MATLAB toolbox and chessboard, and the distortion coefficients are used for correction to obtain the dedistorted image.

[0070] The intrinsic parameters of 4D radar and lidar have been calibrated offline in the factory, which can be divided into two external parameter calibration processes: camera calibration and lidar calibration; 4D radar and lidar external parameter calibration.

[0071] The extrinsic parameters of the camera and 4D radar can be obtained by performing matrix operations on the remaining two extrinsic parameters. The extrinsic parameters between different sensors are expressed as translation and rotation matrices.

[0072] For camera and lidar extrinsic calibration, a checkerboard is used to perform 2D-3D alignment of point cloud and image data to complete a rough calibration.

[0073] Then, the extrinsic parameters are manually fine-tuned by static objects such as trees and poles in the environment. For the extrinsic calibration of 4D radar and lidar, it is regarded as a spatial 3D-3D point cloud alignment.

[0074] First, the distance between the two sensors is measured as a rough translation parameter;

[0075] Then multiple corner reflectors are used in space to fine-tune the external parameters.

[0076] Step 103: using the arrival time as a timestamp to align the data, the 4D radar and lidar point clouds are projected into the image to obtain a fused image;

[0077] Specifically, all sensors work under the ROS driver. Since each sensor runs at a different frame rate, the arrival time of the data is used as a timestamp to align the data, and the final 4D radar and lidar point clouds are projected into the image.

[0078] Step 104: performing dual super-resolution learning processing on the fused image to obtain a surgical area image that has both high resolution and detailed semantic information.

[0079] Specifically, dual super-resolution learning (DSRL) can effectively improve the segmentation accuracy without incurring additional computational cost.

[0080] like Figure 2 As shown in the paper, dual super-resolution learning (DSRL) consists of three parts: a semantic segmentation super-resolution module (SSSR), a single image super-resolution module (SISR), and a feature similarity module (FA), which can maintain high-resolution representation and low-resolution input while reducing the computational complexity of the model.

[0081] Moreover, this method can be easily extended to other tasks, showing excellent performance in semantic segmentation and human pose estimation. For CityScapes semantic segmentation, 70% FLOPs can be used while maintaining performance, and mIoU ≥ 2% improvement can be achieved with similar FLOPs.

[0082] For human pose estimation, we can maintain mAP while reducing FLOPs by 30%, and obtain mAP ≥ 2% improvement at the same FLOPs.

[0083] It can improve the performance while maintaining the inference speed; the high-resolution features of SSSR are further enhanced by FA and SISR. Both branches share the same feature extractor, and the SISR branch is optimized with supervision during training and is freely removed from the network at the inference stage, resulting in no cost overhead. This method can easily achieve higher mIoU with similar FLOPs and maintain performance with fewer FLOPs.

[0084] For example, Figure 3 As shown in (a), the semantic segmentation super-resolution module (SSSR) takes an input of 512×1024 and obtains an output of 1024×2048, which is twice as large as the input image. This method can make full use of the input information and obtain more accurate label predictions.

[0085] Said Figure 3 Figure 2 shows the visualization of SSSR and SISR for the same input (0.5x), where (a) input image, (b) SSSR feature visualization, and (c) SISR feature visualization.

[0086] like Figure 4 Shown, (a) Semantic Segmentation Super-Resolution (SSSR) branch; (b) Extending (a) with Single Image Super-Resolution (SISR) branch. The “encoder” shares the feature extractor.

[0087] Since SISR contains more complete structural information than SSSR, feature similarity learning is introduced to guide SSSR to learn high-resolution representations. As shown in Formula 1, FA aims to learn the matrix similarity between SISR and SSSR, where, as shown in Formula 2, the matrix similarity mainly describes the pairwise relationship between pixels. For a W'×H'×C' feature map F, where W'×H' represents the spatial size, the relationship between every two pixels is formulated, so the relationship map contains W'H'×W'H' connections, and Sij represents the relationship between the i-th pixel and the j-th pixel on the feature map F. In order to reduce the training instability caused by the difference in feature distribution between the SISR and SSSR branches, a feature transformation module is attached to the feature map of the SSSR branch before applying the FA loss. The model consists of a 1×1 convolution layer followed by a BatchNor m and a ReLU layer.

[0088]

[0089] Furthermore, it also includes 4D radar components, XR surgical robots, and satellites to form mission-critical IoT slices.

[0090] In the interconnected mission-critical IoT slice, the IP / MPLS-SDN slice network connects the edge cloud and VMs in the spatial computing through SDN; the spatial computing has virtualized servers, and the server's Hypervisor runs a built-in vRouter / vSwitch. The SDN controller is responsible for creating SDN tunnels between the virtual servers and the DCG / W routers. Subsequently, the SDN controller performs mapping between the SDN tunnels and the MPLS L3 VPN to establish a connection between the spatial computing and the edge cloud.

[0091] The interconnected mission-critical IoT slices are formed through Beidou satellite timestamp slices, which transmit periodic data and non-periodic data in slices in units of time slots, and achieve synchronous transmission of data through a time-sensitive network.

[0092] In order to achieve high-precision and high-efficiency data transmission, the present invention adopts a solution combining Beidou satellite timestamp slicing and time-sensitive networking (TSN).

[0093] The BeiDou satellite system can provide high-precision time services. This invention uses this feature to slice different types of data (such as IoT data, high-definition video streams, surgical images, etc.) in time slots, and each slice is marked with a BeiDou satellite timestamp. This slicing method ensures the timing and accuracy of the data, providing a solid foundation for subsequent data processing and analysis.

[0094] In order to achieve efficient transmission of periodic data (such as control information) and non-periodic data (such as high-definition video, surgical images) in the same network, the present invention introduces time-sensitive networking (TSN) technology.

[0095] Through its scheduling mechanism, TSN can ensure that periodic data is accurately transmitted at predetermined time intervals, while allowing non-periodic data to be transmitted in the remaining time. This mechanism not only simplifies network complexity, but also improves the reliability and real-time performance of data transmission.

[0096] Specifically, periodic control data is transmitted via 100M Ethernet (or higher bandwidth networks) to ensure low latency and high reliability, while non-periodic data, such as high-definition video and surgical images, is transmitted via standard Ethernet to meet the requirements of large data volume and real-time performance.

[0097] TSN's unified network transmission method realizes the deep integration of information technology (IT) and operational technology (OT), further improving the economy and maintainability of the entire system. In surgical applications, time slots, as the smallest unit of circuit-switched aggregated information transmission, can effectively support the real-time control and data transmission requirements of surgery. Through the TSN network, key data during the operation (such as the location of surgical instruments, images of the surgical area, etc.) can be transmitted to the surgeon in real time and accurately, providing strong guarantees for the smooth progress of the operation.

[0098] Experimental Results

[0099] To further verify the effectiveness of the present invention, the dual super-resolution learning method is applied to some other architectures, such as PSPNet based on ResNet101, and several compact networks designed for real-time applications, such as DABNet and BiseNet18 based on ResNet. Table 1 shows the segmentation accuracy and model complexity respectively. FLOP is used to evaluate the network complexity calculated at a resolution of 512×1024.

[0100] It can be seen that the present invention can improve the results on networks of different complexity: ESPNetv2, BiseNet, DABNet, PSPNet and DeepLabv3+. For networks using decoders (such as PSPNet), the improvement on the test set is 4.3%, while the baseline is 69.3%. Compared with the distillation method, another effective pipeline is provided to obtain higher performance under the same FLOPs.

[0101] Table 1

[0102]

[0103] Compute accuracy and GFLOPs at the same image resolution. Refers to the method used in knowledge distillation. Indicates that the network has been tested at multiple scales.

[0104] In one embodiment, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0105] Step 101: construct a communication satellite and 4D radar component interconnection data set;

[0106] Step 102: performing internal parameter calibration, external parameter calibration and time calibration on the 4D imaging radar and the laser radar of the 4D radar assembly;

[0107] Step 103: using the arrival time as a timestamp to align the data, the 4D radar and lidar point clouds are projected into the image to obtain a fused image;

[0108] Step 104: performing dual super-resolution learning processing on the fused image to obtain a surgical area image that has both high resolution and detailed semantic information.

[0109] In one embodiment, a computer-readable storage medium is provided, storing a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0110] Step 101: construct a communication satellite and 4D radar component interconnection data set;

[0111] Step 102: performing internal parameter calibration, external parameter calibration and time calibration on the 4D imaging radar and the laser radar of the 4D radar assembly;

[0112] Step 103: using the arrival time as a timestamp to align the data, the 4D radar and lidar point clouds are projected into the image to obtain a fused image;

[0113] Step 104: performing dual super-resolution learning processing on the fused image to obtain a surgical area image that has both high resolution and detailed semantic information.

[0114] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0115] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0116] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A semantic segmentation method based on Starlink 4D radar, characterized in that: The method comprises: Build a data set for interconnection between communication satellites and 4D radar components; Performing internal parameter calibration, external parameter calibration and time calibration on the 4D imaging radar and the laser radar of the 4D radar assembly; The arrival time is used as the timestamp to align the data, and the 4D radar and lidar point clouds are projected into the image to obtain a fused image; The fused image is subjected to dual super-resolution learning processing to obtain an image of the surgical area having both high resolution and detailed semantic information.

2. The semantic segmentation method based on Starlink 4D radar according to claim 1, characterized in that: The communication satellite and 4D radar component interconnection data set is constructed, including a continuous sequence of 4D radar point clouds and 3D annotation content.

3. The semantic segmentation method based on Starlink 4D radar according to claim 1 or 2, characterized in that: It also includes 4D radar components, XR surgical robots, and satellites to form mission-critical IoT slices.

4. The semantic segmentation method based on Starlink 4D radar according to claim 3 is characterized in that: In the interconnected mission-critical IoT slice, the IP / MPLS-SDN slice network connects the edge cloud and VMs in the spatial computing through SDN; the spatial computing has virtualized servers, and the server's Hypervisor runs a built-in vRouter / vSwitch. The SDN controller is responsible for creating SDN tunnels between the virtual servers and the DCG / W routers. Subsequently, the SDN controller performs mapping between the SDN tunnels and the MPLS L3 VPN to establish a connection between the spatial computing and the edge cloud. The interconnected mission-critical IoT slices are formed through Beidou satellite timestamp slices, which transmit periodic data and non-periodic data in slices in units of time slots, and achieve synchronous transmission of data through a time-sensitive network.

5. The semantic segmentation method based on Starlink 4D radar according to claim 4, characterized in that: The dual super-resolution learning includes a semantic segmentation super-resolution module (SSSR), a single image super-resolution module (SISR), and a feature similarity module (FA).

6. The semantic segmentation method based on Starlink 4D radar according to claim 5, characterized in that: The input image is enlarged twice and then output through the semantic segmentation super-resolution module; and a high-resolution image is constructed from the low-resolution input through the single image super-resolution module.

7. The semantic segmentation method based on Starlink 4D radar according to claim 6, characterized in that: The feature similarity module guides SSSR to learn high-resolution representation The feature similarity module learns the matrix similarity between SISR and SSSR. The matrix similarity mainly describes the pairwise relationship between pixels. Among them, W'×H' represents the spatial size, we formulate the relationship between every two pixels, so the relationship graph contains W'H'×W'H' connections, and Sij represents the relationship between the i-th pixel and the j-th pixel on the feature map F.

8. A semantic segmentation device based on Starlink 4D radar, characterized in that: The device comprises: a data set unit, a calibration unit, and an image processing unit; The data set unit is used to construct a communication satellite and 4D radar component interconnection data set; The calibration unit is used to perform internal parameter calibration, external parameter calibration and time calibration on the 4D imaging radar and the laser radar of the 4D radar assembly; The image processing unit is used to align data using the arrival time as a timestamp, and the 4D radar and lidar point clouds are projected into the image to obtain a fused image; it is also used to perform dual super-resolution learning processing on the fused image to obtain a surgical area image that has both high resolution and detailed semantic information.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.