An enhanced synthetic vision system for offshore operations

Abstract

The present invention discloses an enhanced synthetic vision system suitable for offshore operations, including an external flight ocean environment visualization module, a sea condition information visualization module, an optoelectronic image enhancement module and a laser enhanced display module; the external flight ocean environment visualization module reconstructs a virtual flight environment, the sea condition information visualization module digitally displays sea condition information and wave information, the optoelectronic image enhancement module realizes the fusion display of optoelectronic image information and three-dimensional geographic information, and the laser enhanced display module uses a laser rangefinder to perform laser ranging on the target, combines inertial navigation data and photoelectric potential posture data to solve the geographic information of the target, constructs a static obstacle database, and uses OpenGL to highlight the target. The present invention can significantly improve the helicopter flight operation capability in offshore waters under low visibility conditions such as night and fog through effective combination with optoelectronic systems and complementary advantages.

Description

Technical Field

[0001] The invention belongs to the fields of airborne information processing, airborne optoelectronic system technology and integrated vision technology, and relates to an enhanced synthetic vision system suitable for offshore operations. Background Art

[0002] In recent years, the role of helicopters in offshore surveys and maritime rescue has become increasingly important. Due to the particularity of the marine environment, the sea surface is often accompanied by severe flying conditions such as sea fog, sea breeze, and waves. In addition, the characteristics of the marine environment where the sea and sky are integrated lead to a lack of significant positioning reference objects on the sea surface, which puts pilots in a state of tension to a certain extent, thus restricting the operational efficiency of the helicopter. Summary of the invention

[0003] (I) Purpose of the invention

[0004] The purpose of the present invention is to provide an enhanced synthetic vision system suitable for offshore operations, which can provide a clear three-dimensional flight view that is not restricted by the external meteorological environment, provide pilots with a comprehensive situation information, and assist helicopter offshore operations and safe flight.

[0005] (II) Technical solution

[0006] In order to solve the above technical problems, the present invention provides an enhanced synthetic vision system suitable for offshore operations, characterized in that it includes at least an external flight ocean environment visualization module, a sea condition information visualization module, an optoelectronic image enhancement module, a laser enhanced display module and a flight instrument auxiliary module.

[0007] The ocean environment visualization module is used to reconstruct a virtual flight environment, including a three-dimensional ocean flight environment and a two-dimensional electronic chart. The three-dimensional ocean flight environment is reconstructed using OpenGL and GLSL, and the ocean wave effect is reconstructed using the fast Fourier transform algorithm. In order to avoid the influence of redundant color information in the three-dimensional scene on obstacle recognition, this module performs grayscale monochrome processing on the scene, aiming to form a high-contrast, clutter-free background information. The reconstructed three-dimensional scene is fused with the inertial navigation data for display, and a virtual flight viewpoint consistent with the current flight viewpoint is presented in real time. The two-dimensional electronic chart is a preset multi-source information set, constructed using an offline obstacle database, and has the ability to display pre-planned flight paths, reefs, beaches, coastlines and other information, which can be used to assist helicopter low-altitude flight operations, and can also be used for offshore operation units to work in collaboration with helicopter units.

[0008] The sea condition information visualization module includes digital sea condition information and wave information. The digital sea condition signal is displayed through an independent flight information module, including wind force, temperature, humidity, laser permission status, preset obstacle opening status and fusion mode. The wind force, temperature and humidity information are obtained through meteorological satellites and temperature and humidity sensors. The display of wind force includes wind speed and wind direction; the display of temperature adopts standard Celsius with an accuracy of 0.1℃; and the humidity is displayed in percentage. The wave information is used to characterize the wave information caused by sea breeze or helicopter rotor downwash, and simple line symbols are used to indicate the direction and intensity of the waves.

[0009] The optoelectronic image enhancement module is used to realize the fusion display of optoelectronic image information and three-dimensional geographic information, and mainly involves a viewpoint-based registration and fusion algorithm. The registration and fusion algorithm is divided into three steps: position registration, field of view registration and image fusion. Position registration step: The position coordinates of the airborne optoelectronic equipment in the world coordinate system are used as the viewpoint registration base point, and a negative feedback system is formed with the airborne navigation data to correct the three-dimensional virtual viewpoint position in the scene so that it is consistent with the viewpoint registration base point. Field of view registration step: The optoelectronic image is used as input to calculate the aspect ratio r0 of the optoelectronic image. By calculating the aspect ratio r1 of the current three-dimensional scene field of view and the scale factor of the optoelectronic image r0, the projection matrix of the three-dimensional scene is adjusted to ensure that the three-dimensional scene will not be deformed after fusion. Finally, the image fusion algorithm is used to complete the fusion of the three-dimensional scene image and the optoelectronic image.

[0010] The laser enhanced display module is designed to use the laser rangefinder to perform laser ranging on the target, and then combine the inertial navigation data and the photoelectric potential posture data to solve the target's geographic information, construct a static obstacle database, and use OpenGL to highlight the target. The laser enhanced display module includes a laser ranging state display module and a target obstacle marking module. The laser ranging state display module includes the laser ranging state, ranging distance value, target longitude and target latitude. During the laser ranging process, the module provides the pilot with the current target's geographic information in the form of digital characters, and provides data input for the obstacle marking module. The obstacle marking module refers to presenting the target obstacle located by the laser rangefinder in a three-dimensional scene through three-dimensional symbols. The specific steps are as follows: after the obstacle marking module receives the target longitude and latitude solved after laser ranging, it considers the influence of the geographic vertical reference plane, and uses the ray intersection algorithm to solve the target's precise altitude in the three-dimensional world. The longitude, latitude and altitude of the target form a set of spatial coordinates, and OpenGL is used to complete the drawing of the target, and update the obstacle database.

[0011] The flight instrument auxiliary module includes a pitch indicator, an azimuth indicator, a roll indicator, an altitude indicator, a ground speed display, an airspeed display, a vertical speed display, a horizontal line at the center of the field of view, a horizontal line at the center of the rolling field of view, a crosshair and a warning prompt character. During the flight, the flight instrument auxiliary module receives servo data from the integrated navigation system and the electro-optical stabilization system to drive the flight instrument to move in real time. The warning prompt character is in a blanking state by default. When the altitude indicator height display is less than the specified threshold, the altitude indicator height value display character color is highlighted in red. At this time, the warning character WARNING flashes at a specific frequency and is combined with the voice system to give a prompt.

[0012] (III) Beneficial effects

[0013] The enhanced synthetic vision system suitable for offshore operations provided by the above-mentioned technical solution can significantly improve the helicopter flight operation capability in offshore waters under low visibility conditions such as night and fog through effective combination and complementary advantages with the optoelectronic system. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Diagram of the components of an enhanced synthetic vision system for offshore operations;

[0015] Figure 2 This is a schematic diagram of the visual display effect of sea condition information;

[0016] Figure 3 Schematic diagram of photoelectric enhancement effect;

[0017] Figure 4 This is a schematic diagram of laser enhanced display effect;

[0018] Figure 5 This is a schematic diagram of flight instruments;

[0019] Figure 6 This is a schematic diagram of the effect of an enhanced synthetic vision system for offshore operations. DETAILED DESCRIPTION

[0020] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0021] In view of the phenomena in the background technology, the present invention constructs a novel enhanced synthetic vision system for solving flight navigation and operation tasks under the condition of limited offshore visual cues, involving the construction of sea area geographic information, the construction of visual display screens of sea conditions such as waves and sea breezes, enhanced display of television / thermal imaging / laser sensor data, conformal display of virtual flight reference objects, overlay of flight instruments, etc.

[0022] like Figure 1As shown, the enhanced synthetic vision system suitable for offshore operations in this embodiment includes at least a three-dimensional ocean environment visualization module 101, a sea condition information visualization module 102, an optoelectronic image enhancement module 103, a laser enhancement module 104 and a flight instrument auxiliary module 105.

[0023] Figure 2 This is a display example of an ocean environment visualization module, which is used to reconstruct a virtual three-dimensional flight environment, including a three-dimensional ocean environment 201 and a two-dimensional electronic nautical chart 202 .

[0024] The three-dimensional ocean environment 201 uses OpenGL and GLSL shading languages ​​to construct a three-dimensional earth database in the WGS84 coordinate system, and uses fast Fourier transform to simulate the wave effect on the ocean surface. The three-dimensional ocean environment of this example has two display modes: color and grayscale. The color mode is suitable for situations with good external visibility, and the grayscale mode is suitable for situations with limited visual clues. Users can select different display modes by pressing buttons according to actual conditions. The viewpoint transformation cascade matrix is ​​constructed using navigation data and the attitude angle data of the optoelectronic pod, and matrix operations are performed with the virtual viewpoint in the three-dimensional environment to drive the three-dimensional virtual viewpoint to be consistent with the current flight viewpoint, and the external virtual three-dimensional flight environment is presented in real time to form an immersive three-dimensional environment.

[0025] The two-dimensional electronic nautical chart 202 is a preset multi-source information collection, constructed using an offline obstacle database, and capable of displaying information such as pre-planned flight paths, reefs, beaches and coastlines, which can be used to assist helicopter low-altitude flight operations, and can also be used for offshore operation units to work collaboratively with helicopter units.

[0026] The sea condition information visualization module includes digitized sea condition information 203 and wave information 204 .

[0027] The digital sea condition information 203 is a comprehensive information display panel, including wind force, temperature and humidity, and the above information is obtained through meteorological satellites and temperature and humidity sensors. The display of wind force WIND includes speed and direction, such as 2.5m / sNW, the wind speed is 2.5m / s, NW means northwest wind, the font is Calibri, and the color is green; the display of temperature TEMP adopts standard Celsius, with an accuracy of 0.1℃, and the display is 19.8℃; humidity HUMIDITY is displayed in percentage, such as 12%. The laser permission state Laser defaults to the OFF state, which is not allowed. When the laser permission state is ON, laser ranging operation can be performed. The preset obstacle state Preset defaults to the ON state. At this time, the system will display the preset obstacle information. In the case of good visibility, if the preset obstacle display is inconsistent with the image information of the photoelectric sensor, it is easy to cause visual confusion. The preset obstacle display can be turned off by pressing the button. The system fusion mode FUSION indicates the current scene display mode, including: 3D, 3D+TV, 3D+IR, 3D+LASER. The pilot selects different modes through the buttons around the integrated display.

[0028] The wave information 204 uses simple line symbols to represent the wave information (wave direction and strength) caused by sea breeze or helicopter rotor downwash. The specific meanings of the line symbols are: east E, west W, south S, and north N are displayed in text form around the wireframe circle, and an arrow symbol moves inside the circle to indicate the actual direction of the wave; three parallel sine wave symbols represent the strength of the wave, and the amplitude of the sine wave changes with the strength of the wave.

[0029] Figure 3 This is an example of an optoelectronic image enhancement module. The optoelectronic image enhancement module can fuse and display the optoelectronic small field of view image 301 with the three-dimensional scene large field of view image 302 through a registration and fusion algorithm. The optoelectronic images include television and night vision. The optical axes of television images and night vision images are consistent, so the registration and fusion processes of the two are similar, and only the offset vector of the physical installation size needs to be considered. First, the position coordinates of the airborne optoelectronic equipment in the world coordinate system are used as the viewpoint registration base point, and a negative feedback adjustment system is formed with the airborne navigation data to correct the three-dimensional virtual viewpoint position in the scene so that it is consistent with the viewpoint registration base point. Secondly, the optoelectronic image is used as input to calculate the aspect ratio r0 of the optoelectronic image. By calculating the aspect ratio r1 of the current three-dimensional scene field of view and the scale factor of the optoelectronic image r0, the projection matrix of the three-dimensional scene is adjusted to ensure that the three-dimensional scene will not be deformed. Finally, the image fusion algorithm is used to complete the fusion of the three-dimensional scene image and the optoelectronic image.

[0030] Figure 4It is an example of a laser enhancement module, which includes a laser ranging status display 401 and a target obstacle label 406. The laser ranging status display 401 is used to display the current working status of the laser rangefinder, including: laser ranging status 402, laser ranging distance 403, target longitude 404 and target latitude 405. In the default state, the laser ranging status display 401 is in a blanking state. During laser ranging, the laser ranging status 402 will flash at a fixed frequency, indicating that the laser is ranging, and the current target distance value will be displayed after the Distance note. The system will use the ranging distance value and the airborne inertial navigation data to calculate the target longitude 404 and target latitude 405 of the target in the world coordinate system. The target obstacle marking module 406 receives the longitude B and latitude L calculated by laser ranging, and uses the ray intersection algorithm to calculate the precise height H of the target on the surface under the premise of considering the geographic vertical reference plane, constructs the three-dimensional space coordinates (B, L, H), and uses OpenGL to draw the obstacle symbol of the current position. The obstacle is represented by a broken line symbol with an angle of 30 degrees and a yellow color. Under the visual condition of the optoelectronic device, an obstacle database is constructed based on the ranging results. In the three-dimensional scene, the distance value between the current viewpoint and the obstacle within the viewport range is dynamically calculated, and the distance value 407 is displayed in digital characters just above the broken line symbol in red. When the obstacle is less than the specified threshold from the carrier position, the obstacle broken line symbol will change from yellow to red to highlight the threat level of the target obstacle.

[0031] Figure 5 The flight instrument auxiliary module is a display example of a flight instrument auxiliary module, which is a flight status symbol set, including an azimuth indicator 501, a roll indicator 502, a pitch indicator 503, a pitch angle value 504, a roll angle value 505, a horizontal line at the center of the field of view 506, a crosshair 507, a horizontal line at the center of the rolling field of view 508, an altitude indicator 509, a ground speed display 510, an airspeed display 511, a vertical speed display 512 and warning prompt characters 513 and 514.

[0032] The flight instrument module receives onboard navigation data and drives the flight instruments to dynamically present the current flight attitude in real time, making it easier for the pilot to control the aircraft in a timely manner.

[0033] Azimuth indicator 501 indicates the current azimuth angle of the carrier, ranging from -180 degrees to 180 degrees, with 0 degrees pointing to due north. The range in the eastern hemisphere is 0 degrees to 180 degrees, and the range in the western hemisphere is -180 degrees to 0 degrees. The triangle in the lower center of the azimuth scale bar indicates the position of the current azimuth in the scale bar, and the value in the bubble shape in the upper center of the azimuth scale bar indicates the current azimuth angle value.

[0034] The roll indicator 502, the roll angle value 505 and the center horizontal line 508 of the roll field of view represent the roll state of the carrier aircraft, wherein the roll indicator 502 indicates the direction of the carrier aircraft's roll, the center horizontal line 508 of the roll field of view indicates the actual position of the carrier aircraft's roll, and the roll angle value 505 indicates the current roll angle of the carrier aircraft.

[0035] The pitch indicator 503 and the pitch angle value 504 in combination represent the current pitch angle of the aircraft.

[0036] The field of view center horizontal line 506 represents the horizontal reference line of the current three-dimensional scene.

[0037] The crosshair 507 is located at the geometric center of the three-dimensional scene, indicating the position of the virtual viewpoint of the current three-dimensional scene.

[0038] The altitude indicator 509 receives radio altimeter data, indicating the vertical distance between the current aircraft and the ground. The value of the bubble in the altitude indicator 509 indicates the radio altitude of the current aircraft.

[0039] The ground speed display 510 indicates the current ground speed of the flight, which is displayed as GS 160, wherein GS is the abbreviation of Ground Speed ​​and the number 160 is the current ground speed in km / h.

[0040] The airspeed display 511 indicates the current airspeed of the aircraft, which is displayed as AS 160, where AS stands for Air Speed ​​and the number 160 is the current airspeed in km / h.

[0041] The vertical speed display 512 indicates the current vertical speed of the aircraft, which is displayed as VS 30, wherein VS stands for VerticalSpeed ​​and the number 30 indicates the current vertical speed in km / h.

[0042] The warning prompt character 513 and the warning status mark 514 are combined to represent the warning information of the current flight altitude. The warning prompt character 513 is represented by WARNING, the color is red, and it is in a blank state under the default state. When the altitude indicator 509 data is less than the specified threshold, the warning prompt character 513 and the status mark 514 flash at a specific frequency, and a warning prompt is given in combination with the voice system.

[0043] Figure 6 A working example of an enhanced synthetic vision system suitable for offshore operations.

[0044] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An enhanced synthetic vision system suitable for offshore operations, characterized in that: It includes an external flight ocean environment visualization module, a sea condition information visualization module, an optoelectronic image enhancement module and a laser enhanced display module; the external flight ocean environment visualization module reconstructs a virtual flight environment, the sea condition information visualization module digitally displays sea condition information and wave information, the optoelectronic image enhancement module realizes the fusion display of optoelectronic image information and three-dimensional geographic information, and the laser enhanced display module uses a laser rangefinder to perform laser ranging on the target, combines inertial navigation data and photoelectric potential attitude data to calculate the target's geographic information, constructs a static obstacle database, and uses OpenGL to highlight the target; The enhanced synthetic vision system also includes: a flight instrument auxiliary module, which receives servo data from the integrated navigation system and the electro-optical stabilization system during flight and drives the real-time movement of the flight instruments; The virtual flight environment reconstructed by the external flight ocean environment visualization module includes a three-dimensional ocean flight environment, which is reconstructed using OpenGL and GLSL, and the ocean wave effect is reconstructed using the fast Fourier transform algorithm, and the scene is grayed and monochrome processed. The reconstructed three-dimensional scene is fused with the inertial navigation data for display, and a virtual flight viewpoint consistent with the current flight viewpoint is presented in real time; The virtual flight environment reconstructed by the external flight ocean environment visualization module also includes a two-dimensional electronic nautical chart, which is constructed using an offline obstacle database to display pre-planned flight paths, sea reefs, beaches and coastline information, assist helicopter low-altitude flight operations, and provide for coordinated work between offshore operation units and helicopter units; The sea condition information is displayed through an independent flight information module, including wind force, temperature, humidity, laser permission status, preset obstacle opening status and fusion mode. The wind force, temperature and humidity information are obtained through meteorological satellites and temperature and humidity sensors; the wind force display includes wind speed and wind direction; the temperature display adopts standard Celsius with an accuracy of 0.1°C; the humidity is displayed in percentage; Using line symbols to represent the sea wave information caused by sea breeze or helicopter rotor downwash, the sea wave information includes the direction and intensity of the sea wave; The optoelectronic image enhancement module adopts a viewpoint-based registration and fusion algorithm to fuse optoelectronic image information with three-dimensional geographic information for display. The registration and fusion algorithm is divided into three steps: position registration, field of view registration and image fusion; position registration step: using the position coordinates of the airborne optoelectronic equipment in the world coordinate system as the viewpoint registration base point, forming a negative feedback system with the airborne navigation data, correcting the three-dimensional virtual viewpoint position in the scene to make it consistent with the viewpoint registration base point; field of view registration step: using the optoelectronic image as input, calculating the aspect ratio of the optoelectronic image, and adjusting the projection matrix of the three-dimensional scene by calculating the proportional factor of the aspect ratio of the current three-dimensional scene field of view and the aspect ratio of the optoelectronic image to ensure that the three-dimensional scene does not deform or distort after fusion; finally, using the image fusion algorithm to complete the fusion of the three-dimensional scene image and the optoelectronic image.

2. The enhanced synthetic vision system for offshore operations according to claim 1, characterized in that: The laser enhanced display module includes a laser ranging state display module and a target obstacle marking module; the laser ranging state display module displays the laser ranging state, ranging distance value, target longitude and target latitude. During the laser ranging process, the module provides the pilot with the geographical information of the current target in the form of digital characters, and provides data input for the obstacle marking module; the target obstacle marking module presents the target obstacle located by the laser rangefinder in a three-dimensional scene through three-dimensional symbols.

3. The enhanced synthetic vision system for offshore operations according to claim 2, characterized in that: After receiving the target longitude and latitude calculated by laser ranging, the obstacle marking module considers the influence of the geographic vertical reference plane and uses the ray intersection algorithm to calculate the precise altitude of the target in the three-dimensional world. The longitude, latitude and altitude of the target form a set of spatial coordinates. OpenGL is used to complete the drawing of the target and update the obstacle database.

4. The enhanced synthetic vision system for offshore operations as claimed in claim 3, characterized in that: The flight instrument auxiliary module includes a pitch indicator, an azimuth indicator, a roll indicator, an altitude indicator, a ground speed display, an airspeed display, a vertical speed display, a horizontal line at the center of the field of view, a horizontal line at the center of the rolling field of view, a crosshair and a warning prompt character; the warning prompt character is in a blank state by default. When the altitude display of the altitude indicator is less than a specified threshold, the color of the altitude indicator altitude value display character is highlighted red. At this time, the warning character WARNING flashes at a specific frequency and is prompted in combination with the voice system.

Images