A method and device for networking observations on the sea surface

Abstract

The present invention discloses a method and device for observing a sea surface network. The method includes: a method for observing a sea surface network, connecting multiple observation devices in a target sea area to a background server to form an observation network, and the background server processes the original observation images into expression images in a unified form and stitches them into a stitched image of the target sea area according to the observation positions and outputs it to the server. By unifying the resolution of the observation images of the observation devices and networking them, the present application enables observers to achieve global or regional observation of the target sea area through means such as map zooming and moving. All the image pixels presented to the user server are unified, the expression of sea conditions is clear and definite, saving the time for users to obtain and interpret the images of each observation device. The networked observation output improves the utilization efficiency of the observation devices and greatly facilitates the ocean observation process.

Description

Technical Field

[0001] The present invention belongs to the technical field of ocean observation, and in particular relates to a method and device for networking observation on the sea surface. Background Art

[0002] Oceanography is a science that has gradually developed based on ship surveys and ocean observations. The phenomena involved cover a spatial scale from hundreds of meters to thousands of kilometers and a temporal scale from several hours to several centuries. In terms of current ocean observation means, although there are already remote sensing satellites as high-altitude observation means, limited by image clarity, weather, clouds, satellite orbit time position, etc., the detailed scope of ocean observation is still supplemented and tracked in real time through equipment such as manned / unmanned observation aircraft for aerial photography.

[0003] At the present stage, due to the differences in models and shooting methods of various observation remote sensors such as satellites and drones, there are also significant differences in the resolution of the finally output observation data, especially observation images. For example, satellite observations and drone cameras are different in terms of observation area, detail display, etc. And at the present stage, the observation equipment is all point shooting, and users can only connect to each observation equipment separately to obtain observation images, and cannot obtain the target sea area image as intuitively as on the ground. Figure 1 Therefore, networked observation has always been an important development direction for future ocean observation. Many journals and magazines have proposed to develop methods for large-scale networked observation of the ocean in the future. However, the problems of a wide variety of observation equipment, mismatched imaging pixel ratios, and inability to effectively integrate still exist. Summary of the Invention

[0004] In order to make up for the defects that the images of the above-mentioned observation equipment are different and can only be transmitted point-to-point, the present application proposes a method and device for networking observation on the sea surface. The method includes: a method for networking observation on the sea surface, connecting multiple observation devices in the target sea area to a background server to form an observation network. The background server processes the original observation images into expression images in a unified form and stitches them into a stitched image of the target sea area and outputs it to the server. The method includes: the background server determines the number, distribution, observation image form, and output form of the expression image of the networking observation devices according to the information of various observation devices in the target sea area; the background server divides the target sea area into observation grids according to the positions of the networking observation devices, and at least one observation device is provided in each grid; the observation device obtains the observation image of the corresponding grid and transmits it to the background server, and the background server processes the original observation image into a unified form and stitches it into a stitched image of the target sea area and outputs it to the server.

[0005] Further, the image in the unified form is an image with a unified size, pixel, and resolution.

[0006] Further, the process of processing and converting the original observation image includes: extracting the contour lines of the observation elements from the original observation image, converting the contour lines into vector graphics and magnifying them to the size of the expression image; inputting the magnified vector graphics and the original observation image into a neural network to generate the expression image.

[0007] Further, the contour line extraction method includes: converting the original observation image into a grayscale image, obtaining the grayscale value of each pixel point, and counting the number of pixel points under each grayscale value. A grayscale-pixel curve function is constructed with the grayscale value as the X-axis and the number of pixel points under each grayscale value as the Y-axis. The extraction interval of the contour line grayscale range is determined according to the function slope and the corresponding number of pixel points for extraction.

[0008] Further, the method for determining the tolerance range and interval of the contour line extraction includes: obtaining the slope values of each point of the grayscale-pixel curve function, using the points with slope values in the first third and satisfying that the total number of pixel points within 3-5 grayscale ranges before and after is not less than one percent of the total number of pixel points as anchor points, and using the 3-5 pixel range before and after this point as the extraction interval.

[0009] Further, when the distance between two extraction intervals is less than 6 pixels, the union of the two extraction intervals is used as the final extraction interval.

[0010] Further, the resolution of the expression image is the same as that of the image with the highest number of pixels in the observation image.

[0011] Further, the neural network also includes a pre-training process, and the pre-training process uses historical observation images of the target sea area as the training set and the contour lines of the historical observation images as the guiding conditions for training.

[0012] This application also proposes a networked observation device, including a background server for processing data, a plurality of observation devices connected to the background server and providing observation images, and a server for providing services to users. The networked observation device observes the ocean using any of the above methods.

[0013] This application unifies the resolution of the observation images of the observation devices and networks them, enabling observers to achieve global or regional observation of the target sea area through means such as map zooming and moving. All the images presented to the user server have unified pixel counts, clearly and explicitly expressing the sea conditions. It saves the time for users to obtain images from each observation device and interpret the images. The networked observation output improves the utilization efficiency of the observation devices and greatly facilitates the ocean observation process. Description of the Drawings

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0015] Figure 1 Flow schematic diagram of an implementation manner of the method of the present invention;

[0016] Figure 2 Schematic diagram of the gray - pixel function of an embodiment of the method of the present invention;

[0017] Figure 3 Schematic diagram of the observed image of an embodiment of the method of the present invention

[0018] Figure 4 is Figure 3 Schematic diagram of contour line extraction;

[0019] Figure 5 is Figure 3 Converted expression diagram. Detailed implementation manners

[0020] The following will clearly and completely describe the technical solutions of the present application in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0021] In the description of the present application, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0022] In the description of the present application, it should be noted that unless otherwise clearly specified and limited, the terms "installation", "connection", and "linkage" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.

[0023] The present invention will be further described in detail below with reference to the accompanying drawings.

[0024] Combined with Figure 1 , an implementation manner of the present application. A method for observing a sea surface network. Multiple observation devices in a target sea area are networked with a background server to form an observation network. After the background server obtains the observation images, it converts the observation images into expression images in a unified form and then outputs them to the server. The method specifically includes: the background server determines the number, distribution, form of the observation images, and output form of the expression images according to the information of various observation devices in the target sea area; the background server divides the target sea area into observation grids according to the positions of the networked observation devices, and at least one observation device is provided in each grid; the observation device obtains the observation images of the corresponding grid and transmits them to the background server, and the background server processes the original observation images, converts them into expression images in a unified form, and splices them into a mosaic image of the target sea area according to the observation positions and outputs them to the server. In this embodiment, all available observation devices in the target sea area are networked, and each observation device is comprehensively utilized to observe the ocean. In this embodiment, the main innovation is to facilitate the use of users. The image formats obtained by each observation device are normalized, and an observation map is spliced. During the use of users, it is possible to rotate the angle, change the position, track the path, etc. in a similar way to an electronic map to observe a certain phenomenon. The resolution sizes of the observation images spliced into the electronic map are the same after processing, which is convenient for users to observe the entire sea area, rather than the original local observation only through a single observation device. Figure 1 In the above embodiment, the unified-form image is an image with a unified pixel resolution in terms of size. In this embodiment, for each observation device, in order to facilitate the subsequent splicing into a complete image during the process of unifying the form, an image with a unified pixel resolution in terms of size is adopted, and the resolution of the observation images is unified according to the size of the observation area, so that the ocean scale corresponds to the image scale, in order to reduce the misalignment of the images caused by inconsistent resolutions during the subsequent splicing process.

[0025] Based on the above one or more embodiments, with reference to

[0026] Figures 3 - 5 ​The process of processing and converting the original observation image includes: extracting the contour lines of the observation elements from the original observation image, converting the contour lines into vector graphics and magnifying them to the size of the expression image; inputting the magnified vector graphics and the original observation image into a neural network to generate the expression image. In this embodiment, since this application is mainly applied to the field of observation rather than art, the processed graphics need to be consistent with the details of the original observation image. Therefore, this application retains the main details in the original image as contour lines, and makes up for the ocean background in the image through machine learning to improve the image resolution while ensuring the consistency of the details between the original observation image and the processed image. In this embodiment, the neural network used to generate the expression image needs to use the historical observation images of the target sea area as the training set and the contour lines extracted from the historical observation images as the guiding conditions for training, so that the neural network has the ability to identify the corresponding sea conditions according to the contour lines.

[0027] Based on one or more of the above embodiments, the contour line extraction method includes: converting the original observation image into a grayscale image, obtaining the grayscale value of each pixel point, and counting the number of pixel points at each grayscale value, and combining Figure 4 , taking the grayscale value as the X-axis and the number of pixel points at each grayscale value as the Y-axis to construct a grayscale-pixel curve function, and determining the extraction interval of the contour line grayscale range according to the function slope and the corresponding number of pixel points for extraction. In the field of ocean observation technology, since ocean images are usually simpler than land images, most of the observation contents are waves, islands, reefs, color changes at the boundary between deep and shallow seas, and individual ships. Therefore, the extraction intervals set by PS (PHOTOSHOP) or other matte extraction software during the contour extraction process will cause some objects, such as islands and reefs, with large contrasts with the sea surface to have overly rich details, while waves and the like are omitted due to small contrasts. Therefore, this application uses the grayscale-pixel point extraction method when extracting contour lines. Generally speaking, the expression images are consistent within the same grayscale value range on the sea surface, such as the blue sea, white waves, and darker-colored reefs and islands. In this embodiment, a coordinate system is established with the grayscale value from 0 to 255 as the X-axis and the number of pixel points at each grayscale value as the Y-axis. The slope between adjacent grayscale values and the number of pixel points are used to judge the grayscale boundary according to the smoothness of the grayscale-pixel curve, and then the grayscale extraction range is set to achieve the extraction of the contour line, avoiding over-extraction or omission of objects or sea conditions in individual grayscale value ranges.

[0028] Based on one or more of the above embodiments, combining Figure 4, the method for determining the tolerance range and interval of contour line extraction includes: obtaining the slope values of each point of the gray-pixel curve function, using the points with slope values in the first one-third and satisfying that the total number of pixel points within the range of 3-5 gray levels before and after is not less than one percent of the total number of pixel points as anchor points, and using the range of 3-5 pixels before and after this point as the extraction interval. In this embodiment, the width of 6-10 pixels before and after the anchor point is set as the width of the contour line, and the width of the sampling interval is set according to the pixels of the observation image, ensuring that the line contours can be effectively extracted from the images of instruments with pixel numbers ranging from 300,000 pixels (600*600) to 1,000,000 pixels (1000*1000) in the sea observation equipment without causing the lines to be too wide. In this embodiment, it is limited that the slope value is in the first one-third and the total number of pixel points within the range of 3-5 gray levels before and after is not less than one percent of the total number of pixel points, ensuring that each anchor point belongs to a region where the gray level has a relatively obvious change and there is an approximate gray image, excluding the influence of scattered floating objects or noise points. In this embodiment, generally there are multiple gray anchor points extracted, and multiple contour lines can be extracted according to the wave traces, showing the details of the observation image as much as possible. For example, if multiple gray values such as 59, 82, 126 meet the requirements of the anchor points, then all the pixel points in the observation image with the above gray values and the image range with a radius of 3-5 pixels centered on the corresponding pixel points are extracted. In most cases, the above image range will extend into a contour line expressing the sea conditions.

[0029] Based on the above one or more embodiments, when the distance between two extraction intervals is less than 6 pixels, the union of the two extraction intervals is used as the final extraction interval. In this embodiment, since some waves cross during propagation, the union extraction method is adopted in this embodiment to ensure that the sea conditions shown in the observation image are extracted to the greatest extent.

[0030] Based on the above one or more embodiments, the resolution of the expression image is the same as that of the image with the highest pixels in the observation image. In this embodiment, the high-resolution image is used as the final output image as much as possible to make the expression image clear and show the most details.

[0031] Based on the above one or more embodiments, the neural network further includes a pre-training process, which uses the historical observation images of the target sea area as the training set and the line contours of the historical observation images as the guiding conditions for training. In this embodiment, the neural network is trained through the historical observation images and the line contours of the historical observation images to improve the sensitivity of the neural network to the sea conditions in the corresponding area and the accuracy of image generation in this sea area.

[0032] The present application also provides a networking observation device, which includes a background server for processing data, a plurality of observation devices connected to the background server and providing observation images, and a server for providing services to users. The networking observation device observes the ocean by using the observation method described in any one of the above embodiments.

[0033] By unifying the resolution of the observation images of the observation devices and networking them, the present application enables observers to perform full-domain or regional observations on the target sea area by means such as map zooming and moving. All the image pixels presented to the user server are unified, clearly and explicitly expressing the sea conditions. It saves the time for users to obtain and interpret the images of each observation device. The networking observation output improves the utilization efficiency of the observation devices and greatly facilitates the ocean observation process.

Claims

1. A sea surface networking observation method, wherein multiple observation devices in a target sea area are connected to a backend server to form an observation network, the backend server processes the original observation image, converts it into a unified form of expression image, and splices it into a spliced ​​image of the target sea area according to the observation position and outputs it to the server, characterized in that: The method comprises: a backend server determines the number and distribution of observation equipment, the form of observation images and the output form of expression images according to information of various observation equipment in the target sea area; the backend server divides the target sea area into observation grids according to the observation position, and each grid is provided with at least one observation equipment; the observation equipment obtains the observation image of the corresponding grid and transmits it to the backend server, and the backend server processes the original observation image, converts it into an expression image in a unified form, and splices it into a spliced ​​image of the target sea area and outputs it to the server; The processing and conversion process of the original observation image includes: extracting the line contour of the observation element according to the original observation image, converting the line contour into a vector diagram and enlarging it to the size of the expression image; inputting the enlarged vector diagram and the original observation image into a neural network to generate an expression image; The neural network also includes a pre-training process, wherein the pre-training process uses the historical observation images of the target sea area as a training set and the line contours of the historical observation images as a guiding condition for training; The line contour extraction method comprises: converting the original observed image into a grayscale image and obtaining the grayscale value of each pixel point and counting the number of pixels at each grayscale value, constructing a grayscale-pixel curve function with the grayscale value as the X-axis and the number of pixels at each grayscale value as the Y-axis, and determining the extraction interval of the grayscale range of the line contour according to the function slope and the corresponding number of pixels for extraction.

2. The method according to claim 1, characterized in that The image in the uniform form is an expression image with uniform pixel size and resolution.

3. The method according to claim 1, characterized in that: When the distance between two extraction intervals is less than 6 pixels, the union of the two extraction intervals is taken as the final extraction interval.

4. A networked observation device, comprising a background server for processing data, a plurality of observation devices connected to the background server and providing observation images, and a server for providing services to users, characterized in that: The networked observation device uses the observation method described in any one of claims 1 to 3 to observe the ocean.

Images