Unmanned ship sensing system interference injection method and device
By generating and injecting interference scenario data using an embedded GPU computing card in the unmanned surface vessel (USV) system, the high cost and high risk issues of USV perception systems in real-world environments are solved, enabling efficient and safe testing and verification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-18
- Publication Date
- 2026-03-10
AI Technical Summary
Reproducing rain, snow, fog, noise, and various complex interference conditions in real-world environments is difficult and expensive, resulting in high-cost, high-risk, and non-reproducible testing of unmanned surface vessel perception systems.
By connecting the embedded GPU computing card between the sensor and the unmanned vessel's perception system, interference scene data is generated and injected in real time to simulate complex environments, including fog, rain, snow and noise models, and to construct a high-fidelity digital harsh environment.
It enables safe and efficient extreme testing of the robustness and reliability of perception algorithms in the laboratory, reducing testing costs and improving the repeatability and accuracy of testing.
Smart Images
Figure CN121634052A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of interference processing technology, and in particular to an interference injection method and apparatus for an unmanned vessel sensing system. Background Technology
[0002] In the context of injecting interference into unmanned surface vessel (USV) perception systems at the data level, the primary driving force is to solve the challenges of high cost, high risk, and non-reproducibility in real-world testing. Especially in real-world field environments, reproducing rain, snow, fog, noise, and various complex interference conditions is both difficult and expensive, and fraught with uncertainty. Summary of the Invention
[0003] The purpose of this application is to provide a method and apparatus for interfering injection into an unmanned surface vessel (USV) sensing system, which can safely, efficiently and accurately inject interference regardless of the environment.
[0004] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for injecting interference into an unmanned surface vessel (USV) sensing system, including: Acquire real-world scene data collected by sensors; Collect the interference scenarios configured by the user, and call the corresponding interference model from the preset interference model library according to the interference scenarios to interfere with the real scene data and obtain the interference scene data; The disturbed scene data is sent to the unmanned vessel perception system in real time.
[0005] Secondly, this application provides an interference injection device for an unmanned vessel sensing system, including an embedded GPU computing card; the embedded GPU computing card executes a computer program to implement an interference injection method for an unmanned vessel sensing system. The embedded GPU computing card serves as a data middleware, connected in series between the sensor and the unmanned vessel's perception system.
[0006] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application combines a hardware-in-the-loop embedded architecture, setting up an embedded GPU computing card between the real sensors and the unmanned vessel perception system to inject interference into real-world scene data in real time and accurately, simulating various user-configured interference scenarios. This allows researchers to safely and efficiently complete extreme stress tests and systematic evaluations of the robustness and reliability of perception algorithms in the laboratory without relying on uncontrollable and costly real-world field tests. In short, this application constructs a high-fidelity, quantitatively reproducible "digital harsh environment," fundamentally revolutionizing the testing and verification paradigm of unmanned vessel perception systems. Attached Figure Description
[0007] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0008] Figure 1 This is a flowchart illustrating the interference injection method for an unmanned vessel sensing system in one embodiment of this application.
[0009] Figure 2 This is a schematic diagram of an interference injection device for an unmanned vessel sensing system in one embodiment of this application. Detailed Implementation
[0010] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0011] The technological foundation of this application includes: first, hardware-based interference data injection; and second, perturbation algorithms, such as generative adversarial networks (GANs), capable of generating extremely realistic interference data. These technologies collectively ensure that the injected interference conforms to physical laws while possessing sufficient diversity, realism, and real-time performance. This application not only significantly accelerates the research and iteration of perception algorithms, honing their robustness by covering massive amounts of extreme scenarios, but also systematically evaluates the reliability bottlenecks of the entire device, providing crucial technical support for building an "all-weather" reliable autonomous unmanned surface vessel system adapted to the complex challenges of the real world.
[0012] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0013] In one exemplary embodiment, such as Figure 1 As shown, an interference injection method for an unmanned surface vessel (USV) sensing system is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. This application is applicable to everyday weather (e.g., sunny days); it is not required in windy, rainy, or snowy weather. The embodiments of this application include steps 101 to 103.
[0014] Step 101: Acquire real scene data collected by the sensor; the real scene data includes video streams from the camera and point cloud data packets from the LiDAR.
[0015] Step 102: Collect the interference scenario configured by the user, and call the corresponding interference model from the preset interference model library according to the interference scenario to interfere with the real scene data, thereby obtaining the interfered scene data. Specifically, according to the user-preset interference scenario, the corresponding interference model is called to modify or overlay the original data in real time.
[0016] The preset interference model library includes meteorological interference models and noise and distortion models; the meteorological interference models include fog models and rain / snow models.
[0017] In one specific application, the fog model is used for: (1) Receive video stream and extract any frame of digital image from the video stream; generally, a clear digital image in RGB format is obtained with pixel values ranging from 0 to 255.
[0018] (2) Normalize the digital image.
[0019] (3) Based on the atmospheric scattering model, the scattering of light in fog is simulated. By adjusting the concentration and visibility, the normalized digital image is subjected to color feature change processing and spatial feature change processing. Among them, the color feature change processing includes changes in overall hue, saturation, and contrast; the spatial feature change processing includes changes in edge sharpness, texture detail processing, and depth perception.
[0020] Specifically, color feature transformation processing is applied to the digital image, including shifting the overall hue towards atmospheric light color; reducing overall saturation, making the color appear grayish-white; significantly reducing contrast, and blurring details. Simultaneously, spatial feature transformation processing is applied to the image, including edge sharpness (edges become blurred, high-frequency information is lost); texture details are reduced as fine textures are obscured by fog effects; depth perception is enhanced, conforming to atmospheric perspective principles. The transmittance calculation formula is as follows: t(x) = exp(-β × d(x)); t(x) is the transmittance at position x in the digital image (0-1); β is the scattering coefficient, positively correlated with fog density; d(x) is the depth value in the depth map corresponding to the digital image.
[0021] In one specific application, the rain / snow model is used for: (1) Receive video stream and extract any frame of digital image from the video stream; generally, a clear digital image in RGB format is obtained with pixel values ranging from 0 to 255.
[0022] (2) By adjusting the precipitation type, precipitation intensity and wind direction and speed, the precipitation and accumulation effect of raindrops / snowflakes are simulated on the digital image to change the visual effect characteristics of the digital image; wherein, the precipitation type includes rain, snow and sleet, which is used to determine the shape of raindrops or snowflakes; the precipitation intensity includes at least light rain, moderate rain, heavy rain and rainstorm, which is used to determine the density of raindrops or snowflakes; the wind direction and speed are used to determine the direction of raindrops or the trajectory of falling snowflakes.
[0023] Specifically, for rain / snow precipitation particles, the raindrop model uses ellipsoidal or teardrop shapes, with a diameter of 0.5mm-5mm (corresponding to 1-10 pixels). The raindrop density varies according to precipitation intensity (50 drops / frame - 5000 drops / frame), and the raindrop direction is influenced by wind direction. The snowflake model uses hexagonal crystals, with a diameter of 1mm-15mm (corresponding to 2-30 pixels). The snowflake density varies according to snowfall intensity (20 flakes / frame - 2000 flakes / frame), and the snowflakes move through random rotation and falling trajectories.
[0024] The rain / snow model generates visual effect changes based on the image: rain effects are characterized by striped patterns, slanted linear raindrop trails, and fog effects that enhance the air humidity of rainy days. Snow effects are characterized by flocculent patterns and irregular snowflake shapes; the colors become brighter, and the overall brightness of the scene increases.
[0025] In a specific application, the noise and distortion model is used for: (1) Receive video stream and point cloud data packets, and extract any frame of digital image from the video stream.
[0026] (2) First determine the mean parameter of the noise (usually set to 0, indicating zero mean noise) through the Gaussian noise model, and then set the standard deviation parameter of the noise to control the noise intensity.
[0027] (3) Based on the mean parameter and standard deviation parameter of the noise, add the corresponding noise value to each pixel of the digital image or each point data of the point cloud data packet, limit the data within the effective range of the sensor, and keep the statistical characteristics of the data basically unchanged.
[0028] Adding noise results in image quality degradation in digital images, producing granular or snowflake-like visual noise. For point data, the outline of the point cloud becomes blurred.
[0029] Step 103: The disturbed scene data is sent to the unmanned vessel's perception system in real time. At this point, the "contaminated" data stream is sent to the unmanned vessel's perception and computing unit in real time, making it "believe" that it is in a real, harsh environment.
[0030] Based on the same inventive concept, this application also provides an apparatus. The solution provided by this apparatus is similar to the solution described in the above method. Therefore, the specific limitations of one or more apparatus embodiments provided below can be found in the limitations of the method above, and will not be repeated here.
[0031] In one exemplary embodiment, an interference injection device for an unmanned surface vessel (USV) sensing system is provided, including an embedded GPU computing card. The embedded GPU computing card executes a computer program to implement the aforementioned interference injection method for the USV sensing system, and ensures the real-time performance of data processing by running a complex interference generation algorithm.
[0032] The embedded GPU computing card, acting as a data middleware, is connected in series between the actual physical sensors and the unmanned vessel's perception system. Thus, the device of this application constitutes a hardware-in-the-loop testing device. It does not generate physical interference but rather serves as a data middleware, connected in series between the unmanned vessel's physical sensors (cameras, LiDAR, etc.) and the unmanned vessel's perception system (USV), which includes perception computing units such as industrial control computers / AI computing cards, capable of outputting perception results such as detection / recognition results with interference.
[0033] The embedded GPU computing card is equipped with input and output interfaces, which correspond to... Figure 2 The system includes a high-speed data interface for input and output; the input interface supports multiple sensor interfaces, including at least Ethernet (camera, LiDAR), CAN / FD (radar, AIS), and serial port (GPS / IMU); the output interface corresponds to the input interface and outputs the processed data stream to the sensing computing unit.
[0034] This application also manages the data flow and task scheduling of the entire device through a main control MCU. That is, the GPU handles cognitive computing, while the MCU focuses on reflective control. For example... Figure 2 As shown, the device also includes: Human-computer interaction interface (corresponding) Figure 2 The control and configuration interface (in the system) is electrically connected to the embedded GPU computing card and is used to: collect user-configured interference scenarios and / or user-defined interference parameters; the interference parameters include: concentration and visibility in the fog model, precipitation type, precipitation intensity and wind direction and speed in the rain / snow model, and mean parameters and standard deviation parameters in the noise and distortion model.
[0035] In practical applications, the human-computer interaction interface can be a touchscreen, allowing users to select preset scenarios (such as "dense fog Level 4" or "moderate rain + ocean waves Level 3") or customize interference parameters with a single click. It also allows users to drag and drop different interference models, set their intensity and duration, and save them as complex test cases.
[0036] The power supply and heat dissipation components, connected to the embedded GPU computing card, are used to provide power and dissipate heat. That is, they provide a stable power supply and are equipped with an efficient cooling system to ensure stable operation over extended periods.
[0037] In one specific application, the embedded GPU computing card is used for: (1) After receiving the real scene data collected by the sensor, the RTSP / RTP protocol is parsed and the compressed video stream data packets are separated. Specifically, for the video stream, the network receiving and decryption protocol is executed first: the receiving application (such as using the `libavformat` library of FFmpeg) connects to the input RTSP stream, parses the RTSP / RTP protocol, and separates the compressed video data packets (such as H.264 / H.265 bitstream).
[0038] (2) Hardware decoding of the compressed video stream data packets yields the original YUV frames. Specifically, the compressed video stream data packets are sent to the NVDEC, a dedicated decoding hardware on the GPU, which decodes them into original YUV frames. These decoded original frames are then stored in the GPU's video memory (VRAM) instead of being read back into CPU memory. This avoids expensive memory copying and is key to achieving high performance.
[0039] (3) Collect the interference scene configured by the user, and call the corresponding interference model from the preset interference model library according to the interference scene to interfere with the original YUV frame and obtain the interference scene data.
[0040] (4) The interfered scene data is hardware encoded, then repackaged and pushed to the unmanned vessel perception system via the RTSP server protocol stack. Specifically, the interfered scene data (still in the GPU memory) is directly sent to NVENC, a dedicated encoding hardware on the GPU, which recompresses it into a new video format (such as H.264 or HEVC). Repackaging and streaming encapsulate the compressed video data packets encoded by NVEC into a suitable format (such as MPEG-TS) and pushes it out via the RTSP server protocol stack to form a new RTSP output stream.
[0041] The above processing constitutes a highly efficient software framework, ensuring that the sensor data stream can pass through the interference model and be injected with extremely low latency. For video streams, the latency can be controlled at the millisecond level. For point cloud data streams, the device directly acquires data point information, including the (x, y, z) spatial coordinates of each point, and applies noise processing to the acquisition time information of each point before outputting it.
[0042] In summary, this application constructs a high-fidelity, quantitatively reproducible "digital harsh environment," fundamentally revolutionizing the testing and verification paradigm of unmanned surface vessel (USV) perception systems. Through a hardware-in-the-loop embedded architecture, real-time and precise interference injection is performed on the data stream between the real sensors and computing units, simulating various scenarios ranging from common rain and fog to extreme sea clutter and even malicious attacks. This allows researchers to safely and efficiently conduct extreme stress tests and systematic evaluations of the robustness and reliability of perception algorithms in the laboratory, without relying on uncontrollable and costly real-world field tests.
[0043] The direct result is a significant acceleration of the iterative upgrade and reliability certification process for the intelligence level of unmanned surface vessels (USVs). By quantitatively analyzing the performance degradation curves of the perception system under different interference intensities, algorithmic bottlenecks can be accurately located and optimized and strengthened accordingly. Ultimately, this provides indispensable technical support for forging an "all-weather" autonomous USV system that can operate stably and safely in real, complex marine environments, significantly improving the maturity and reliability of the final product.
[0044] This application enables the injection of simulated interference into the unmanned vessel perception system via hardware in both laboratory software environments and field real-vehicle environments, thereby achieving safe, efficient, and repeatable testing and verification.
[0045] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0046] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for injecting interference into an unmanned surface vessel (USV) sensing system, characterized in that, The method comprises: acquiring real scene data collected by a sensor; collecting an interference scene configured by a user, and calling a corresponding interference model from a preset interference model library according to the interference scene to interfere with the real scene data, so as to obtain post-interference scene data; real-time sending the post-interference scene data to an unmanned ship perception system.
2. The unmanned ship perception system jamming injection method of claim 1, wherein, The real scene data comprises a video stream from a camera component and a point cloud data packet from a laser radar. The interference models in the preset interference model library comprise a meteorological interference model and a noise and distortion model; the meteorological interference model comprises a fog model and a rain / snow model.
3. The unmanned ship perception system jamming injection method of claim 2, wherein, The fog model is used for: receiving a video stream and extracting any frame digital image from the video stream; performing normalization processing on the digital image; based on an atmospheric scattering model, simulating scattering of light in fog, and performing color feature change processing and spatial feature change processing on the normalized digital image by adjusting concentration and visibility. The color feature change processing comprises changes in overall hue, saturation and contrast; and the spatial feature change processing comprises changes in edge sharpness, texture details and depth perception.
4. The unmanned ship perception system jamming injection method of claim 2, wherein, The rain / snow model is used for: receiving a video stream and extracting any frame digital image from the video stream; by adjusting precipitation type, precipitation intensity and wind direction and speed, simulating raindrop / snowflake precipitation and accumulation effects on the digital image to change visual effect features of the digital image. The precipitation type comprises rain, snow and sleet, and is used to determine raindrop shape or snowflake shape; the precipitation intensity comprises at least light rain, moderate rain, heavy rain and storm, and is used to determine raindrop density or snowflake density; and the wind direction and speed are used to determine raindrop direction or snowflake falling trajectory.
5. The unmanned ship perception system jamming injection method of claim 2, wherein, The noise and distortion model is used for: receiving a video stream and a point cloud data packet, and extracting any frame digital image from the video stream; by a Gaussian noise model, first determining a mean value parameter of noise, and then setting a standard deviation parameter of noise to control noise intensity; according to the mean value parameter and the standard deviation parameter of the noise, adding a corresponding noise value to each pixel point of the digital image or each point data of the point cloud data packet.
6. An unmanned ship perception system jamming injection apparatus, comprising: The device comprises an embedded GPU computing card; the embedded GPU computing card executes a computer program to implement the unmanned ship perception system interference injection method of any one of claims 1-5; the embedded GPU computing card is used as data middleware and is connected in series between a sensor and the unmanned ship perception system.
7. The uncrewed vessel perception system interference injection apparatus of claim 6, wherein, The embedded GPU computing card is provided with an input interface and an output interface; the input interface supports multiple sensor interfaces, at least including Ethernet, CAN / FD and serial port; and the output interface corresponds to the input interface.
8. The uncrewed vessel perception system interference injection apparatus of claim 6, wherein, The device further comprises: a human-computer interaction interface, which is electrically connected to the embedded GPU computing card and is used to collect an interference scene configured by a user and / or user-defined interference parameters; the interference parameters comprise concentration and visibility in the fog model, precipitation type, precipitation intensity and wind direction and speed in the rain / snow model, and mean value parameter and standard deviation parameter in the noise and distortion model.
9. The uncrewed vessel perception system interference injection apparatus of claim 6, wherein, The device further comprises: The power supply and heat dissipation component is connected with the embedded GPU computing card, and is used for providing power supply and heat dissipation.
10. The uncrewed vessel perception system jamming injection apparatus of claim 6, wherein, The embedded GPU computing card is used for: After receiving the real scene data collected by the sensor, the RTSP / RTP protocol is parsed, and the compressed video stream data packet is separated out; The compressed video stream data packet is hardware decoded to obtain the original YUV frame; The user configured interference scene is collected, and a corresponding interference model is called from a preset interference model library according to the interference scene, so as to interfere with the original YUV frame to obtain the interference scene data; The interference scene data is hardware encoded, and then is re-encapsulated and pushed to the unmanned ship sensing system through the RTSP server protocol stack.