Shoreline identification processing method and system based on artificial intelligence

Abstract

The embodiment of the invention relates to the technical field of marine monitoring, and discloses an artificial intelligence-based shoreline identification processing method, which comprises the following steps of: extracting a key frame image in real-time video stream data, and carrying out preprocessing operation on the obtained key frame image; identifying the preprocessed key frame image based on a pre-constructed shoreline identification model to extract a feature parameter result of a corresponding shoreline, updating the key frame image according to the feature parameter result to obtain a shoreline identification image, and storing coordinates of each pixel point in the shoreline identification image; and mapping the corresponding pixel points in the coastline identification image to the remote sensing image according to the mapping relationship between the corresponding pixel point coordinates and the corresponding point locations and the mapping relationship between the corresponding remote sensing pixel coordinates and the corresponding point locations for overlapping display. According to the method provided by the embodiment of the invention, the video monitoring data and the remote sensing image data are combined, and multi-source data fusion is realized.

Description

Technical Field

[0001] The present invention relates to the field of ocean monitoring technology, and in particular to a shoreline identification and processing method and system based on artificial intelligence. Background Art

[0002] At present, the ocean observation and monitoring system is an important application of modern science and technology in ocean research and management. These systems use various sensors, satellite remote sensing, drones and other equipment to monitor the marine environment in real time, providing valuable data for marine scientific research, environmental protection, resource development and disaster warning. Coastal terrain identification is an important task of the ocean observation and monitoring system, which is of great significance for coastal engineering, port construction, marine resource development and disaster prevention and mitigation. Although the existing monitoring system has played a role in coastal terrain identification, it still has some defects. For example, satellite remote sensing data is limited by weather conditions and satellite orbit period, and real-time monitoring cannot be achieved; although drone aerial photography has high resolution, it is costly and inefficient when applied on a large scale; ground measurement is time-consuming and labor-intensive, and is not suitable for frequent monitoring; and due to the limitations of sensor accuracy, data processing capabilities and algorithm models, existing systems often find it difficult to accurately and quickly identify coastal terrain, especially under complex terrain and dynamic environmental conditions. Therefore, designing a solution that can improve the information processing capability of the monitoring system has become a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0003] In view of the above-mentioned defects, an embodiment of the present invention discloses a shoreline recognition and processing method based on artificial intelligence, which can provide more comprehensive and accurate terrain information.

[0004] The first aspect of the embodiment of the present invention discloses a shoreline recognition and processing method based on artificial intelligence, comprising:

[0005] Acquire real-time video stream data of the shoreline area through a video monitoring system set up in the corresponding coastal area, extract key frame images from the real-time video stream data, and perform preprocessing operations on the obtained key frame images;

[0006] Based on the pre-built shoreline recognition model, the pre-processed key frame image is recognized to extract the characteristic parameter results of the corresponding shoreline, and the key frame image is updated according to the characteristic parameter results to obtain a shoreline recognition image, and the coordinates of each pixel point in the shoreline recognition image are saved;

[0007] Selecting corresponding pixel coordinates in the coastline recognition image, and determining a mapping relationship between the corresponding pixel coordinates and the corresponding point based on the pixel coordinates and a preset pixel-point mapping table;

[0008] Acquire a remote sensing image of the corresponding shoreline area, and determine a mapping relationship between the corresponding remote sensing pixel coordinates and the corresponding point based on the remote sensing pixel coordinates of the remote sensing image and a pre-set remote sensing-point mapping table;

[0009] According to the mapping relationship between the corresponding pixel point coordinates and the corresponding point positions and the mapping relationship between the corresponding remote sensing pixel coordinates and the corresponding point positions, the corresponding pixel points in the coastline recognition image are mapped to the remote sensing image for overlay display.

[0010] As an optional implementation manner, in the first aspect of the embodiment of the present invention, after extracting the key frame image in the real-time video stream data, the method further includes:

[0011] Determine an image timestamp corresponding to a key frame image in real-time video stream data;

[0012] After the pre-processed key frame image is identified based on the pre-built shoreline identification model to extract the characteristic parameter result of the corresponding shoreline, the method further includes:

[0013] Acquire tide height data at a corresponding time and a corresponding position according to the image timestamp, and determine the terrain height parameter of the corresponding shoreline according to the tide height data and the characteristic parameter result of the corresponding shoreline;

[0014] The obtained terrain height parameters corresponding to the location of the shoreline are processed into data grids, and the contour value of each grid point is calculated;

[0015] Draw a contour map based on the contour values ​​of each grid point and output the corresponding contour map.

[0016] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the identification processing method further includes:

[0017] The coastline position and terrain height parameters corresponding to each moment are obtained, and the coastline change information of the coastline is analyzed according to the coastline position and terrain height parameters corresponding to each moment, wherein the coastline change information includes the coastline shape change, direction change and height change.

[0018] As an optional implementation, in the first aspect of the embodiment of the present invention, the pre-processed key frame image is recognized based on the pre-built shoreline recognition model to extract the characteristic parameter results of the corresponding shoreline, and the key frame image is updated according to the characteristic parameter results to obtain a shoreline recognition image, and the coordinates of each pixel point in the shoreline recognition image are saved, including:

[0019] Based on the pre-built shoreline recognition model, the pre-processed key frame image is recognized to generate a mask image, wherein the mask image is used to mark the areas of each category in the key frame image;

[0020] Filter out the pixel points corresponding to the shoreline category, perform interpolation processing on the breakpoints of the shoreline, and remove the outliers of the shoreline to form a continuous shoreline;

[0021] The formed continuous coastline is redrawn on the original image, and the coordinate information of each pixel on the coastline is saved.

[0022] As an optional implementation, in the first aspect of the embodiment of the present invention, the shoreline recognition model is constructed based on an improved U-Net model, the U-Net model includes an encoder, a decoder and an output layer, the encoder is a residual encoder, the residual encoder uses a residual module as a basic module, and the residual module includes a convolution layer, a batch normalization layer and an activation layer;

[0023] The method of identifying the preprocessed key frame image based on the pre-built shoreline recognition model to extract the characteristic parameter results of the corresponding shoreline includes:

[0024] The preprocessed key frame image is input into the encoder part for feature extraction;

[0025] In the residual module, the input image is processed by the residual module to obtain the output feature map;

[0026] The extracted feature map is input into the axis attention mechanism for enhancement. The axis attention mechanism decomposes the feature map into two axes, row and column, and calculates row attention and column attention respectively.

[0027] The feature map enhanced by the axis attention mechanism then enters the channel attention mechanism for optimization;

[0028] After being processed by the encoder, axis attention mechanism, and channel attention mechanism, the feature map is passed to the decoder part of U-Net for size recovery and detail reconstruction;

[0029] Output a segmentation map with the same size as the input image to determine the characteristic parameter results of the shoreline, where each pixel in the segmentation map is assigned a category label.

[0030] As an optional implementation manner, in the first aspect of the embodiment of the present invention, after the key frame image is updated according to the characteristic parameter result to obtain the coastline recognition image, the method further includes:

[0031] Orthorectification is performed on the coastline identification image to obtain a corrected image.

[0032] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the orthorectifying the coastline recognition image to obtain a corrected image includes:

[0033] The coastline recognition image is input, and control points with known positions are selected on the coastline recognition image, wherein the control points have corresponding known three-dimensional coordinates on actual land features;

[0034] Calculating parameters in a collinearity equation using the image coordinates and the three-dimensional coordinates of the control points, wherein the collinearity equation is used to characterize the relationship between the image coordinates and the world coordinates;

[0035] Performing perspective transformation on the coastline recognition image according to a collinearity equation to obtain a perspective transformed image;

[0036] The image obtained after the perspective transformation is orthorectified to obtain an orthorectified image.

[0037] A second aspect of an embodiment of the present invention discloses a shoreline recognition and processing system based on artificial intelligence, comprising:

[0038] The first acquisition module is used to acquire real-time video stream data of the coastline area through a video monitoring system set up in the corresponding coastal area, extract key frame images from the real-time video stream data, and perform preprocessing operations on the obtained key frame images;

[0039] Extraction module: used to identify the pre-processed key frame image based on the pre-built shoreline recognition model to extract the characteristic parameter results of the corresponding shoreline, and update the key frame image according to the characteristic parameter results to obtain a shoreline recognition image, and save the coordinates of each pixel point in the shoreline recognition image;

[0040] A selection module is used to select the corresponding pixel coordinates in the coastline recognition image, and determine the mapping relationship between the corresponding pixel coordinates and the corresponding point based on the pixel coordinates and a preset pixel-point mapping table;

[0041] The second acquisition module is used to acquire the remote sensing image of the corresponding shoreline area, and determine the mapping relationship between the corresponding remote sensing pixel coordinates and the corresponding point based on the remote sensing pixel coordinates of the remote sensing image and the preset remote sensing-point mapping table;

[0042] Overlay display module: used to map the corresponding pixel points in the coastline recognition image to the remote sensing image for overlay display according to the mapping relationship between the corresponding pixel point coordinates and the corresponding point positions and the mapping relationship between the corresponding remote sensing pixel coordinates and the corresponding point positions.

[0043] The third aspect of an embodiment of the present invention discloses an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the artificial intelligence-based shoreline identification and processing method disclosed in the first aspect of the embodiment of the present invention.

[0044] A fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the artificial intelligence-based shoreline identification and processing method disclosed in the first aspect of an embodiment of the present invention.

[0045] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0046] The method in the embodiment of the present invention combines video monitoring data and remote sensing image data to achieve the fusion of multi-source data. Video monitoring data provides high-resolution, real-time shoreline change information, while remote sensing images provide a wider spatial coverage and long-term series data. The combination of the two makes the monitoring and analysis of shoreline changes more comprehensive and in-depth. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 It is a flow chart of a shoreline identification and processing method based on artificial intelligence disclosed in an embodiment of the present invention;

[0049] Figure 2 It is a schematic diagram of the process of outputting a contour map disclosed in an embodiment of the present invention;

[0050] Figure 3 It is a schematic diagram of the process of performing image recognition disclosed in an embodiment of the present invention;

[0051] Figure 4 is a schematic diagram of a process for image correction disclosed in an embodiment of the present invention;

[0052] Figure 5 is a schematic diagram of a residual module disclosed in an embodiment of the present invention;

[0053] Figure 6 It is a schematic diagram of the specific structure of the improved model disclosed in the embodiment of the present invention;

[0054] Figure 7It is a structural schematic diagram of a shoreline identification and processing system based on artificial intelligence provided by an embodiment of the present invention;

[0055] Figure 8 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] It should be noted that the terms "first", "second", "third", "fourth", etc. in the specification and claims of the present invention are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having" in the embodiments of the present invention and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0058] Although the existing monitoring system has played a role in coastal terrain identification, there are still some defects. For example, satellite remote sensing data is limited by weather conditions and satellite orbit periods, and real-time monitoring cannot be achieved; although drone aerial photography has high resolution, it is costly and inefficient when applied on a large scale; ground measurement is time-consuming and labor-intensive, and is not suitable for frequent monitoring; and due to the limitations of sensor accuracy, data processing capabilities, and algorithm models, existing systems often have difficulty in accurately and quickly identifying coastal terrain, especially under complex terrain and dynamic environmental conditions. Based on this, the embodiment of the present invention discloses a shoreline identification and processing method, system, electronic device, and storage medium based on artificial intelligence, which combines video monitoring data and remote sensing image data to achieve multi-source data fusion. Video monitoring data provides high-resolution, real-time shoreline change information, while remote sensing images provide a wider spatial coverage and long time series data. The combination of the two makes the monitoring and analysis of shoreline changes more comprehensive and in-depth.

[0059] Embodiment 1

[0060] See also Figure 1 , Figure 1It is a flow chart of the shoreline identification and processing method based on artificial intelligence disclosed in the embodiment of the present invention. Among them, the execution subject of the method described in the embodiment of the present invention is an execution subject composed of software and / or hardware, and the execution subject can receive relevant information by wired or / and wireless means, and can send certain instructions. Of course, it can also have certain processing functions and storage functions. The execution subject can control multiple devices, such as a remote physical server or cloud server and related software, or it can be a local host or server and related software that performs related operations on a device placed somewhere. In some scenarios, multiple storage devices can also be controlled, and the storage devices can be placed in the same place or different places as the devices. For example Figures 1 to 6 As shown, the artificial intelligence-based shoreline recognition and processing method includes the following steps:

[0061] S101: acquiring real-time video stream data of a coastline area through a video monitoring system arranged in a corresponding coastline area, extracting key frame images from the real-time video stream data, and performing a preprocessing operation on the obtained key frame images;

[0062] S102: Recognize the preprocessed key frame image based on the pre-built coastline recognition model to extract the characteristic parameter results of the corresponding coastline, update the key frame image according to the characteristic parameter results to obtain a coastline recognition image, and save the coordinates of each pixel point in the coastline recognition image;

[0063] S103: Selecting corresponding pixel coordinates in the coastline recognition image, and determining a mapping relationship between the corresponding pixel coordinates and corresponding points based on the pixel coordinates and a preset pixel-point mapping table;

[0064] S104: Acquire a remote sensing image of the corresponding shoreline area, and determine a mapping relationship between the corresponding remote sensing pixel coordinates and the corresponding point based on the remote sensing pixel coordinates of the remote sensing image and a preset remote sensing-point mapping table;

[0065] S105: Mapping corresponding pixel points in the coastline recognition image onto the remote sensing image for overlay display according to the mapping relationship between corresponding pixel point coordinates and corresponding point positions and the mapping relationship between corresponding remote sensing pixel coordinates and corresponding point positions.

[0066] The solution of the embodiment of the present invention can obtain the video stream data of the shoreline area in real time through the video monitoring system set up in the coastal area, and extract the key frame image for preprocessing, so as to realize the instant capture of the shoreline changes. Based on the pre-built shoreline recognition model, the characteristic parameters of the shoreline can be extracted efficiently and accurately, thereby generating a shoreline recognition image. Compared with traditional manual monitoring or low-frequency satellite remote sensing, this method has higher real-time and accuracy.

[0067] The entire processing flow is highly automated, from the acquisition of video data, the extraction and preprocessing of key frame images, to the extraction of shoreline features and the generation of shoreline recognition images, to the mapping of pixels and points, and finally the overlay display with remote sensing images. It all relies on artificial intelligence technology and pre-built models, which greatly reduces human intervention and improves processing efficiency and intelligence.

[0068] Through the correspondence between pixel coordinates and point mapping tables, both the pixel points in the coastline recognition image and the pixel points in the remote sensing image can be accurately located at specific points. This precise spatial positioning capability is of great significance in the fields of coastline change monitoring, disaster warning, environmental protection, etc.

[0069] This method combines video monitoring data and remote sensing image data to achieve the fusion of multi-source data. Video monitoring data provides high-resolution, real-time shoreline change information, while remote sensing images provide a wider spatial coverage and long-term series of data. The combination of the two makes the monitoring and analysis of shoreline changes more comprehensive and in-depth. By mapping the pixel points in the shoreline identification image to the remote sensing image for overlay display, the shoreline changes can be intuitively displayed, providing decision makers with intuitive and accurate shoreline change information, which is helpful to formulate scientific and reasonable coastal management policies and measures.

[0070] More preferably, after extracting the key frame image in the real-time video stream data, the method further includes:

[0071] Determine an image timestamp corresponding to a key frame image in real-time video stream data;

[0072] like Figure 2 As shown, after the pre-processed key frame image is identified based on the pre-built shoreline recognition model to extract the characteristic parameter result of the corresponding shoreline, it also includes:

[0073] S1021: Acquire tide height data at a corresponding time and a corresponding position according to the image timestamp, and determine a terrain height parameter at a location corresponding to the coastline according to the tide height data and a characteristic parameter result of the corresponding coastline;

[0074] S1022: performing data gridding processing on the terrain height parameters corresponding to the location of the shoreline, and calculating the contour value of each grid point;

[0075] S1023: Draw a contour map according to the contour value of each grid point, and output the corresponding contour map.

[0076] By determining the image timestamp corresponding to the key frame image in the real-time video stream data, the embodiment of the present invention can accurately associate the shoreline recognition result with the specific time point. Furthermore, obtaining the tide height data at the corresponding time and position according to the image timestamp helps to correct the influence of tidal changes on the shoreline recognition result and improve the accuracy and reliability of shoreline feature parameter extraction.

[0077] The embodiment of the present invention combines the tide height data and the shoreline characteristic parameter results to determine the terrain height parameters corresponding to the shoreline location. This step not only takes into account the horizontal position of the shoreline, but also introduces information in the vertical direction (i.e., terrain height), making the description of the shoreline more complete and accurate. The terrain height parameters are gridded for data processing, which can facilitate spatial analysis and calculation. The calculation of the contour value of each grid point provides basic data for drawing contour maps. Contour maps are an important way to represent terrain, which can intuitively show the ups and downs and changing trends of terrain, and are of great significance to coastal terrain analysis, disaster risk assessment and other fields.

[0078] More preferably, the identification processing method further includes:

[0079] The coastline position and terrain height parameters corresponding to each moment are obtained, and the coastline change information of the coastline is analyzed according to the coastline position and terrain height parameters corresponding to each moment, wherein the coastline change information includes the coastline shape change, direction change and height change.

[0080] The embodiment of the present invention can realize real-time monitoring of dynamic changes of the coast by continuously acquiring the coastline position and terrain height parameters at each moment. This real-time monitoring capability is crucial for timely detection of changes such as coastal erosion, siltation, retreat or advance, and helps to take timely measures to protect coastal resources and environment.

[0081] By analyzing the changes in the location of the coastline and its terrain height parameters, we can obtain comprehensive information on the shape, direction and height changes of the coastline. This information not only reflects the physical changes in the coastline, but also reveals the evolution of natural processes such as coastal geology, geomorphology and hydrology, providing a scientific basis for coastal management and planning. By analyzing coastal change information, we can predict and assess the disaster risks that the coastal zone may face, such as storm surges, coastal erosion, and sea level rise. This early warning and assessment capability helps to take measures in advance to reduce disaster losses and protect people's lives and property.

[0082] More preferably, the pre-processed key frame image is identified based on the pre-built coastline identification model to extract the characteristic parameter results of the corresponding coastline, and the key frame image is updated according to the characteristic parameter results to obtain a coastline identification image, and the coordinates of each pixel point in the coastline identification image are saved, including:

[0083] Based on the pre-built shoreline recognition model, the pre-processed key frame image is recognized to generate a mask image, wherein the mask image is used to mark the areas of each category in the key frame image;

[0084] Filter out the pixel points corresponding to the shoreline category, perform interpolation processing on the breakpoints of the shoreline, and remove the outliers of the shoreline to form a continuous shoreline;

[0085] The formed continuous coastline is redrawn on the original image, and the coordinate information of each pixel on the coastline is saved.

[0086] The embodiment of the present invention generates a mask image to mark the regions of each category in the key frame image, and can clearly separate the shoreline region from other regions, thereby more accurately extracting the characteristic parameters of the shoreline. In addition, interpolation processing is performed on the breakpoints of the shoreline and outliers are removed to form a continuous shoreline, further improving the accuracy of shoreline recognition.

[0087] Redrawing the formed continuous coastline on the original image can generate a clear coastline recognition image. This image not only helps to observe and analyze the morphology and changes of the coastline more intuitively, but also provides high-quality input for subsequent image processing and analysis. Saving the coordinate information of each pixel on the coastline can conveniently store and manage coastline data. These data can be used for subsequent applications such as coastline change monitoring, disaster warning, and resource planning, providing a scientific basis for coastal management and planning. The above process uses automated image processing and recognition technology to achieve accurate identification and feature extraction of the coastline, reducing manual intervention and errors. This helps to improve the automation and intelligence level of coastline recognition technology and improve processing efficiency and accuracy.

[0088] More preferably, the shoreline recognition model is constructed based on an improved U-Net model, the U-Net model includes an encoder, a decoder and an output layer, the encoder is a residual encoder, the residual encoder uses a residual module as a basic module, and the residual module includes a convolution layer, a batch normalization layer and an activation layer;

[0089] like Figure 3 As shown, the pre-processed key frame image is identified based on the pre-built shoreline recognition model to extract the characteristic parameter results of the corresponding shoreline, including:

[0090] S102a: inputting the preprocessed key frame image into the encoder part for feature extraction;

[0091] S102b: In the residual module, the input image is processed by the residual module to obtain an output feature map;

[0092] S102c: The extracted feature map is input to the axis attention mechanism for enhancement. The axis attention mechanism decomposes the feature map into two axes, row and column, and calculates row attention and column attention respectively.

[0093] S102d: The feature map enhanced by the axis attention mechanism then enters the channel attention mechanism part for optimization;

[0094] S102e: After being processed by the encoder, axis attention mechanism, and channel attention mechanism, the feature map is passed to the decoder part of U-Net for size recovery and detail reconstruction;

[0095] S102f: Output a segmentation map with the same size as the input image to determine the characteristic parameter results of the shoreline, wherein each pixel in the segmentation map is assigned a category label.

[0096] The embodiment of the present invention adopts a residual encoder and utilizes the convolution layer, batch normalization layer and activation layer in the residual module. The model can extract image features more deeply and alleviate the gradient vanishing problem in the deep network, thereby improving the accuracy of shoreline recognition. The introduction of the axis attention mechanism and the channel attention mechanism further enhances the model's ability to capture image features, especially for targets such as shorelines with specific shapes and texture features. The model can more accurately identify their boundaries and details.

[0097] The design of the residual encoder makes it easier for the model to learn deep features in the image during training, which helps improve the generalization ability of the model and maintains stable recognition performance when facing images under different lighting, angles, occlusion and other conditions. The combined use of the axis attention mechanism and the channel attention mechanism enables the model to more comprehensively understand the contextual information in the image, further enhancing its adaptability to complex scenes. The batch normalization layer in the residual module helps to speed up the training process of the model, while reducing the model's sensitivity to initialization parameters, making the model easier to converge.

[0098] The structural characteristics of the U-Net model enable it to gradually reduce the size of the feature map during the encoding process, and gradually restore the size during the decoding process. This structure helps to reduce the amount of calculation and improve the operating efficiency of the model. Since the model optimizes the calculation efficiency while maintaining high accuracy, it can support real-time shoreline monitoring. This is of great significance for application scenarios such as real-time monitoring of shoreline changes and disaster warning.

[0099] Specifically, the basic u-net model is improved to identify the shoreline, resnet is used as the encoder to extract features, and ResBlock is used as the basic module of resnet, which includes two groups of (convolution + batch norm + RELU) combinations and shortcuts. Figure 5 As shown, H(x)=F(x)+x, where H(x) is the output of the residual, x is the input, and before the residual part, F(x) is the function of the residual unit, which is used to map the input (x) and generate an activation output.

[0100] After each layer of feature extraction or upsampling, the axis attention mechanism and channel attention mechanism are added. The specific structure is as follows Figure 6 As shown in the figure, the axis attention mechanism decomposes the image into two axes, rows and columns. There are two main forms of axis attention mechanism: row attention and column attention. In tasks such as image processing, the row attention mechanism focuses on the vertical direction of the image, while the column attention mechanism focuses on the horizontal direction. By performing self-attention in the vertical and horizontal directions of the image respectively, the amount of computation can be reduced, especially when processing large images.

[0101] The calculation method of self-attention is:

[0102] Q=X×W Q , K = X × W K , V = X × W V

[0103]

[0104] Where X is the input matrix, W Q , W K , W V are the weight matrices for query, key, and value respectively, Y is the output matrix, and D is the matrix dimension.

[0105] The Softmax function can be expressed as:

[0106] Among them, Y is the output category, × is the input data, ai is the original output value of the i-th output unit, and K is the total number of categories.

[0107] The function of the Softmax function is to convert the output of the neural network into a probability distribution for multi-classification tasks. By converting the original output value through an exponential function, the Softmax function makes the output value of each output unit between (0, 1), and the sum of the output values ​​of all output units is 1, which meets the requirements of probability distribution.

[0108] The calculation method of the axis attention mechanism, taking the row axis as an example, is:

[0109] Q row =X×W Q , K row =X×W K , V row =X×W V

[0110]

[0111] Y=Attention row ×V row

[0112] Where X is the input matrix with dimension n×m, W Q , W K , W V are the weight matrices for query, key, and value, respectively, and their dimensions are m×d k 、m×d k and m×d v , Y is the output matrix, d k d v is the row axis matrix dimension, then Q row , K row The dimension is n×d k , V row The dimension is n×d v , Q row ×K T row The dimension is n×n. The row axis attention mechanism calculates the vertical attention and calculates the attention for all rows in each column. row .

[0113] Channel attention mechanism Squeeze-and-Excitation (SE) Block: It consists of two parts: Squeeze and Excitation. Squeeze: Use global average pooling to compress a H×W×C feature map into 1×1×C. In this way, the global information of each channel is obtained. Excitation: The C-dimensional feature vector obtained by the Squeeze operation is processed by convolution and activation function to readjust the weights between channels. Specifically, the weight coefficient of each channel is generated through two fully connected layers FC and two activation functions Relu and Sigmoid activation functions.

[0114] After the Excitation operation, the generated weight coefficients are weighted and summed with the original feature map to obtain the final feature map. This process can enhance the feature representation capability and improve the performance of the model.

[0115] The channel attention mechanism can help the model better understand multi-channel information, and by automatically selecting and strengthening the information of important channels, the model can more effectively process complex visual information. This helps the model find important information faster while ignoring unimportant information.

[0116] The ReLU (Rectified Linear Unit) activation function is f((x) = max(0, x). When the input value is greater than 0, the output of the ReLU function is equal to the input value; when the input value is less than or equal to 0, the output of the ReLU function is 0. The Sigmoid activation function is:

[0117] More preferably, after the key frame image is updated according to the characteristic parameter result to obtain the coastline recognition image, the method further includes:

[0118] Orthorectification is performed on the coastline identification image to obtain a corrected image.

[0119] More preferably, Figure 4 As shown, the orthorectifying the coastline recognition image to obtain a corrected image includes:

[0120] S106: inputting the coastline recognition image, and selecting control points of known positions on the coastline recognition image, wherein the control points have corresponding known three-dimensional coordinates on actual land features;

[0121] S107: Calculate parameters in a collinearity equation using the image coordinates and the three-dimensional coordinates of the control points, where the collinearity equation is used to characterize the relationship between the image coordinates and the world coordinates;

[0122] S108: performing perspective transformation on the coastline recognition image according to a collinearity equation to obtain a perspective transformed image;

[0123] S109: performing orthorectification on the image obtained after the perspective transformation to obtain an orthorectified image.

[0124] The orthorectification of the embodiment of the present invention can accurately reflect the relationship between the image coordinates and the world coordinates by selecting control points with known positions and using the image coordinates and three-dimensional coordinates of these control points to calculate the parameters in the collinearity equation. This process helps to eliminate image distortion caused by factors such as shooting angle and lens distortion, and improve the spatial accuracy of the coastline recognition image.

[0125] Orthorectified images have higher spatial consistency and accuracy, which makes the images more reliable in subsequent analysis, processing and application. For example, in application scenarios such as coastline change monitoring and coastal resource planning, orthorectified images can provide more accurate spatial information and help make more scientific decisions. Orthorectified images are not only suitable for coastline identification and analysis, but can also be widely used in geographic information systems (GIS), remote sensing image processing, urban planning and other fields. These application scenarios have high requirements for the spatial accuracy and consistency of images, and orthorectification technology can meet these needs.

[0126] Orthorectification as a subsequent step in coastline identification image processing helps to improve the overall image processing process. By identifying coastline features first and then orthorectifying, we can ensure that the final image contains accurate coastline information while meeting the requirements of spatial consistency and accuracy.

[0127] Specifically, the conversion of longitude and latitude of shoreline pixel coordinates requires calibration first, selecting several points on the image, conducting on-site RTK measurement, and using drone scanning to generate point cloud data.

[0128] Orthophoto conversion technology: 1. The surveillance camera converts the light signal received by the sensor into an electrical signal to generate a surveillance image. Therefore, there is a certain correspondence between the pixel coordinates on the surveillance image and the actual location of the object. 2. Since the projection center, image point and target point are all on the light propagation path. In clear weather, the distance between the object and the camera is fixed, and the gas molecules and aerosol content between them are low, the refraction of light is not obvious, and the reflected light directly from the object can basically be used to represent the light signal received at the camera. At this time, the path of the light is a straight line, so the perspective transformation can be used to correct the surveillance image to an orthophoto projection image. 3. By setting known control points (the three-dimensional coordinates of the pixels of the surveillance image and the objects), the parameters in the collinearity equation can be calculated. Since the optical mechanical correction method is mainly selected, it is necessary to select control points on a similar horizontal plane. After calculating the collinearity equation parameters, a fixed correspondence between each pixel and the pixel of the orthophoto correction image can be obtained. 4. Through this fixed correspondence, the real-time surveillance image can be converted into a real-time orthophoto image. By associating the orthophoto with the geodetic coordinate system, the actual position of the objects represented by each pixel can be obtained, thus enabling quantitative analysis.

[0129] Orthorectification algorithm scheme, the orthorectification scheme here is implemented based on perspective transformation. Perspective transformation is to establish a transformation matrix to project the tilted points into regular points. Here, the warpPerspective function of opencv is used to implement it. The coordinates of the points on the image and the converted coordinate points need to be input. Since the images taken by the camera have certain distortion, an image is divided into multiple quadrilaterals. The specific division scheme is based on the actual situation. Then these quadrilaterals are perspective transformed one by one.

[0130] Based on the above technology, the image recognized by artificial intelligence is used as input for correction, and then the corrected image is superimposed on the remote sensing base map scanned by the drone (obtained by processing point cloud data). Since the single-pixel coastline has been redrawn on the recognition result map, the coastline will be corrected and superimposed on the base map during correction. At the same time, the pixel coordinates of the coastline and the pixel coordinates on the remote sensing base map form a one-to-one mapping relationship, which is mapped to the actual measured longitude and latitude coordinates, that is, the relationship between pixel-pixel-latitude and longitude.

[0131] The solution of the embodiment of the present invention extracts the coastline regularly and superimposes it, so that the coastline position at different times can be formed. The coastline information at different times and positions is used as the contour distribution of the terrain, and the terrain elevation is equivalently converted to the tidal height at the corresponding time, and finally a dense distribution is formed. Theoretically, when the time approaches infinity, the distribution formed can form a complete terrain according to different coastline shapes and elevation information, which can be used for three-dimensional visualization, modeling and other terrain expressions, and on this basis, accumulate time series data to realize the prediction of coastline-tide-topography: terrain evolution, tide level and tide height warning and other applications.

[0132] Existing video monitoring systems have achieved remarkable results in tasks such as target detection and region segmentation. However, in three-dimensional scenes, there are still certain limitations for elements such as the height, thickness, and depth of the target that are difficult to obtain directly through plane images. To address this problem, the present invention proposes a shoreline extraction method based on artificial intelligence, which combines actual measured tide data to achieve the drawing of terrain height and contour lines. The following is an overview of the technical solution of the invention:

[0133] Data collection and preprocessing: The real-time video stream of the shoreline area is obtained through the video monitoring system. These video data are preprocessed, including noise reduction, enhancement and other operations, to improve the accuracy of subsequent processing.

[0134] Coastline extraction: Artificial intelligence technology, especially deep learning algorithms, is used to learn from pre-processed video data and automatically extract the characteristics of the coastline. The goal of this step is to identify the location of the coastline in the video and provide basic data for subsequent terrain height and contour drawing.

[0135] Tide data fusion: Collect tide height data at the corresponding time and integrate the tide height information into the shoreline extraction result. Through the tide data, the terrain height of the shoreline can be calculated.

[0136] Topographic contour drawing: Based on the shoreline extraction results integrated with tide data, artificial intelligence technology is used to analyze the change pattern of the shoreline and draw terrain contour lines. The purpose of this step is to intuitively show the height changes of the terrain in the form of contour lines.

[0137] Data conversion and mapping: Convert the calibrated point longitude and latitude to the pixel coordinates of the relative position in the image. The purpose of this step is to accurately map the extracted coastline with the remote sensing base map to ensure the accuracy of the terrain contour distribution.

[0138] Result output: The final terrain contour distribution results are output in a visual form, such as topographic maps, charts, etc., for use in related fields. Through the above technical solution, the present invention can effectively extract and display elements such as target height, thickness, depth, etc. in a three-dimensional scene. This method not only improves the intelligence level of the monitoring system, but also provides more accurate and comprehensive terrain information for coastal engineering, resource development, disaster warning and other fields. At the same time, the present invention has broad application prospects and potential economic value.

[0139] The method in the embodiment of the present invention combines video monitoring data and remote sensing image data to achieve the fusion of multi-source data. Video monitoring data provides high-resolution, real-time shoreline change information, while remote sensing images provide a wider spatial coverage and long-term series data. The combination of the two makes the monitoring and analysis of shoreline changes more comprehensive and in-depth.

[0140] Embodiment 2

[0141] See also Figure 7 , Figure 7 Schematic diagram of the structure of the artificial intelligence-based shoreline recognition and processing system disclosed in the embodiment of the present invention. Figure 7 As shown, the artificial intelligence-based shoreline recognition and processing system may include:

[0142] The first acquisition module 21 is used to acquire real-time video stream data of the coastline area through a video monitoring system set up in the corresponding coastal area, extract key frame images from the real-time video stream data, and perform preprocessing operations on the obtained key frame images;

[0143] Extraction module 22: used to identify the pre-processed key frame image based on the pre-built shoreline recognition model to extract the characteristic parameter results of the corresponding shoreline, and update the key frame image according to the characteristic parameter results to obtain a shoreline recognition image, and save the coordinates of each pixel point in the shoreline recognition image;

[0144] Selection module 23: used for selecting corresponding pixel coordinates in the coastline recognition image, and determining a mapping relationship between the corresponding pixel coordinates and the corresponding point based on the pixel coordinates and a preset pixel-point mapping table;

[0145] The second acquisition module 24 is used to acquire a remote sensing image of a corresponding shoreline area, and determine a mapping relationship between a corresponding remote sensing pixel coordinate and a corresponding point based on the remote sensing pixel coordinates of the remote sensing image and a preset remote sensing-point mapping table;

[0146] The overlay display module 25 is used to map the corresponding pixel points in the coastline recognition image onto the remote sensing image for overlay display according to the mapping relationship between the corresponding pixel point coordinates and the corresponding point positions and the mapping relationship between the corresponding remote sensing pixel coordinates and the corresponding point positions.

[0147] The method in the embodiment of the present invention combines video monitoring data and remote sensing image data to achieve the fusion of multi-source data. Video monitoring data provides high-resolution, real-time shoreline change information, while remote sensing images provide a wider spatial coverage and long-term series data. The combination of the two makes the monitoring and analysis of shoreline changes more comprehensive and in-depth.

[0148] Embodiment 3

[0149] See also Figure 8 , Figure 8 Schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. The electronic device may be a computer, a server, etc. Of course, in certain circumstances, it may also be a smart device such as a mobile phone, a tablet computer, a monitoring terminal, and an image acquisition device with processing functions. Figure 8 As shown, the electronic device may include:

[0150] A memory 510 storing executable program codes;

[0151] a processor 520 coupled to the memory 510;

[0152] The processor 520 calls the executable program code stored in the memory 510 to execute part or all of the steps in the artificial intelligence-based shoreline identification and processing method in the first embodiment.

[0153] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute part or all of the steps in the artificial intelligence-based shoreline identification and processing method in Embodiment 1.

[0154] The embodiment of the present invention further discloses a computer program product, wherein when the computer program product is run on a computer, the computer is enabled to execute part or all of the steps in the artificial intelligence-based shoreline identification and processing method in Embodiment 1.

[0155] An embodiment of the present invention also discloses an application publishing platform, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps in the artificial intelligence-based shoreline identification and processing method in Embodiment 1.

[0156] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0158] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the method described in each embodiment of the present invention.

[0160] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0161] A person of ordinary skill in the art can understand that some or all of the steps in the various methods of the embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0162] The above is a detailed introduction to the artificial intelligence-based shoreline identification and processing method, system, electronic device and storage medium disclosed in the embodiments of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A shoreline recognition and processing method based on artificial intelligence, characterized in that: include: Acquire real-time video stream data of the shoreline area through a video monitoring system set up in the corresponding coastal area, extract key frame images from the real-time video stream data, and perform preprocessing operations on the obtained key frame images; Based on the pre-built shoreline recognition model, the pre-processed key frame image is recognized to extract the characteristic parameter results of the corresponding shoreline, and the key frame image is updated according to the characteristic parameter results to obtain a shoreline recognition image, and the coordinates of each pixel point in the shoreline recognition image are saved; Selecting corresponding pixel coordinates in the coastline recognition image, and determining a mapping relationship between the corresponding pixel coordinates and the corresponding point based on the pixel coordinates and a preset pixel-point mapping table; Acquire a remote sensing image of the corresponding shoreline area, and determine a mapping relationship between the corresponding remote sensing pixel coordinates and the corresponding point based on the remote sensing pixel coordinates of the remote sensing image and a pre-set remote sensing-point mapping table; According to the mapping relationship between the corresponding pixel point coordinates and the corresponding point positions and the mapping relationship between the corresponding remote sensing pixel coordinates and the corresponding point positions, the corresponding pixel points in the coastline recognition image are mapped to the remote sensing image for overlay display.

2. The artificial intelligence-based shoreline recognition and processing method according to claim 1, characterized in that: After extracting the key frame image in the real-time video stream data, the method further includes: Determine an image timestamp corresponding to a key frame image in real-time video stream data; After the pre-processed key frame image is identified based on the pre-built shoreline identification model to extract the characteristic parameter result of the corresponding shoreline, the method further includes: Acquire tide height data at a corresponding time and a corresponding position according to the image timestamp, and determine the terrain height parameter of the corresponding shoreline according to the tide height data and the characteristic parameter result of the corresponding shoreline; The obtained terrain height parameters corresponding to the location of the shoreline are processed into data grids, and the contour value of each grid point is calculated; Draw a contour map based on the contour values ​​of each grid point and output the corresponding contour map.

3. The artificial intelligence-based shoreline recognition processing method according to claim 2, characterized in that: The identification processing method further includes: The coastline position and terrain height parameters corresponding to each moment are obtained, and the coastline change information of the coastline is analyzed according to the coastline position and terrain height parameters corresponding to each moment, wherein the coastline change information includes the change of coastline shape, direction and height.

4. The shoreline recognition processing method based on artificial intelligence according to claim 1, characterized in that: The pre-processed key frame image is identified based on the pre-built coastline identification model to extract the characteristic parameter results of the corresponding coastline, and the key frame image is updated according to the characteristic parameter results to obtain a coastline identification image, and the coordinates of each pixel point in the coastline identification image are saved, including: Based on the pre-built shoreline recognition model, the pre-processed key frame image is recognized to generate a mask image, wherein the mask image is used to mark the areas of each category in the key frame image; Filter out the pixel points corresponding to the shoreline category, perform interpolation processing on the breakpoints of the shoreline, and remove the outliers of the shoreline to form a continuous shoreline; The formed continuous coastline is redrawn on the original image, and the coordinate information of each pixel on the coastline is saved.

5. The artificial intelligence-based shoreline identification and processing method according to claim 1, characterized in that: The shoreline recognition model is constructed based on an improved U-Net model, wherein the U-Net model includes an encoder, a decoder and an output layer, wherein the encoder is a residual encoder, wherein the residual encoder uses a residual module as a basic module, and wherein the residual module includes a convolutional layer, a batch normalization layer and an activation layer; The method of identifying the preprocessed key frame image based on the pre-built shoreline recognition model to extract the characteristic parameter results of the corresponding shoreline includes: The preprocessed key frame image is input into the encoder part for feature extraction; In the residual module, the input image is processed by the residual module to obtain the output feature map; The extracted feature map is input into the axis attention mechanism for enhancement. The axis attention mechanism decomposes the feature map into two axes, row and column, and calculates row attention and column attention respectively. The feature map enhanced by the axis attention mechanism then enters the channel attention mechanism for optimization; After being processed by the encoder, axis attention mechanism, and channel attention mechanism, the feature map is passed to the decoder part of U-Net for size recovery and detail reconstruction; Output a segmentation map with the same size as the input image to determine the characteristic parameter results of the shoreline, where each pixel in the segmentation map is assigned a category label.

6. The artificial intelligence-based shoreline identification and processing method according to claim 1, characterized in that: After the key frame image is updated according to the characteristic parameter result to obtain the coastline recognition image, the method further includes: Orthorectification is performed on the coastline identification image to obtain a corrected image.

7. The artificial intelligence-based shoreline recognition and processing method according to claim 6, characterized in that: The orthorectifying the coastline recognition image to obtain a corrected image includes: The coastline recognition image is input, and control points with known positions are selected on the coastline recognition image, wherein the control points have corresponding known three-dimensional coordinates on actual land features; Calculating parameters in a collinearity equation using the image coordinates and the three-dimensional coordinates of the control points, wherein the collinearity equation is used to characterize the relationship between the image coordinates and the world coordinates; Performing perspective transformation on the coastline recognition image according to a collinearity equation to obtain a perspective transformed image; The image obtained after the perspective transformation is orthorectified to obtain an orthorectified image.

8. A shoreline recognition and processing system based on artificial intelligence, characterized in that: include: The first acquisition module is used to acquire real-time video stream data of the coastline area through a video monitoring system set up in the corresponding coastal area, extract key frame images from the real-time video stream data, and perform preprocessing operations on the obtained key frame images; Extraction module: used to identify the pre-processed key frame image based on the pre-built shoreline recognition model to extract the characteristic parameter results of the corresponding shoreline, and update the key frame image according to the characteristic parameter results to obtain a shoreline recognition image, and save the coordinates of each pixel point in the shoreline recognition image; A selection module is used to select the corresponding pixel coordinates in the coastline recognition image, and determine the mapping relationship between the corresponding pixel coordinates and the corresponding point based on the pixel coordinates and a preset pixel-point mapping table; The second acquisition module is used to acquire the remote sensing image of the corresponding shoreline area, and determine the mapping relationship between the corresponding remote sensing pixel coordinates and the corresponding point based on the remote sensing pixel coordinates of the remote sensing image and the preset remote sensing-point mapping table; Overlay display module: used to map the corresponding pixel points in the coastline recognition image to the remote sensing image for overlay display according to the mapping relationship between the corresponding pixel point coordinates and the corresponding point positions and the mapping relationship between the corresponding remote sensing pixel coordinates and the corresponding point positions.

9. An electronic device, characterized in that: include: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the artificial intelligence-based shoreline identification and processing method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program enables a computer to execute the artificial intelligence-based shoreline identification and processing method as described in any one of claims 1 to 7.

Images