Wave information acquisition method and device, equipment and medium

Abstract

The invention provides a wave information acquisition method and device, equipment and a medium. Performing feature extraction on the to-be-detected image of the target area to obtain a waterline of the to-be-detected image; the to-be-detected image is any frame of image extracted from a to-be-detected video of the target area; acquiring a satellite acquisition image corresponding to the target area, and performing feature matching on the to-be-detected image and the satellite acquisition image to obtain a plurality of matched feature point pairs; adjusting a waterline of the to-be-detected image based on the plurality of feature point pairs and the satellite acquisition image, and determining a target waterline of the to-be-detected image; based on the target waterline of the to-be-detected image, performing image section sequence sampling on the to-be-detected video by adopting a stacking mode to obtain a time stack image of the target area; based on the time stack image, the wave information in the to-be-detected video of the target area is extracted, and the wave information can be extracted more concisely, efficiently and accurately in combination with the time stack image.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular, to a method, apparatus, device, and medium for obtaining wave information. Background Art

[0002] Wave counting, speed, and run-up extraction are core technologies for coastal zone digitization, and are of crucial significance for understanding the marine dynamic environment, predicting coastal erosion, evaluating seawall design, and protecting the coastal ecosystem.

[0003] Accurately measuring wave counting, speed, and run-up is extremely important for disaster prevention and mitigation, assessing the risk of coastal erosion, designing coastal protection facilities, and planning coastal buildings. In the research on coastal zone digitization, through modern technical means such as remote sensing satellites, unmanned aerial vehicles, and camera monitoring, parameters such as wave counting, speed, and run-up can be monitored and analyzed with high precision, providing a scientific basis for coastal management, environmental protection, and disaster warning.

[0004] In the related art, through the installation and calibration of cameras, the acquisition of wave run-up images, and the analysis of image data, the spatial distribution of wave run-up is obtained, and wave run-up data is automatically extracted from the video. However, for areas with low water-land contrast, such as coastal tidal flats and other scenarios, the recognition effect is poor and the wave information extraction is inaccurate.

[0005] Therefore, there is an urgent need for a method for obtaining wave information to extract wave information in the coastal zone. Summary of the Invention

[0006] Embodiments of the present application provide a method, apparatus, device, and medium for obtaining wave information to solve the problem of poor recognition effect and inaccurate wave information extraction in areas with low water-land contrast in the related art.

[0007] In a first aspect, embodiments of the present application provide a method for obtaining wave information, the method comprising:

[0008] Performing feature extraction on a to-be-detected image of a target area to obtain a water edge line of the to-be-detected image; the to-be-detected image is any one frame image extracted from a to-be-detected video of the target area;

[0009] Obtaining a satellite acquisition image corresponding to the target area, and performing feature matching on the to-be-detected image and the satellite acquisition image to obtain a plurality of pairs of matching feature points;

[0010] Based on the plurality of pairs of feature points and the satellite acquisition image, adjusting the water edge line of the to-be-detected image to determine a target water edge line of the to-be-detected image;

[0011] Based on the target water edge line of the image to be detected, the image profile sequence sampling of the video to be detected is carried out in a stacked manner to obtain the time-stack image of the target area;

[0012] Based on the time-stack image, the wave information in the video to be detected in the target area is extracted.

[0013] Based on the above solution, by feature matching of the satellite-acquired image and the image to be detected, a more accurate target water edge line is obtained. Then, based on the target water edge line, the image profile sequence sampling of the video to be detected is carried out to obtain the time-stack image of the target area. Combining with the time-stack image, the extraction of wave information can be carried out more simply, efficiently and accurately.

[0014] In a possible implementation manner, the feature extraction of the image to be detected in the target area to obtain the water edge line of the image to be detected includes:

[0015] Through a semantic segmentation grid, feature extraction of the image to be detected is carried out to obtain the segmentation mask image of the image to be detected;

[0016] The segmentation mask image is vectorized to obtain the water body contour line of the image to be detected;

[0017] The image to be detected is matched with a preset detection area, and the water body contour line located within the preset detection area is used as the water edge line of the image to be detected.

[0018] Based on the above solution, by vectorizing the segmentation mask image of the image to be detected to obtain the water body contour line, and then using the water body contour line located within the preset detection area as the water edge line of the image to be detected, an accurately detected water edge line can be obtained, and the water edge lines with inaccurate edge detection in the image to be detected are screened out.

[0019] In a possible implementation manner, the feature matching of the image to be detected and the satellite-acquired image to obtain multiple pairs of matching feature points includes:

[0020] Feature extraction of the image to be detected is carried out to obtain multiple first feature points of the image to be detected, and feature extraction of the satellite-acquired image is carried out to obtain multiple second feature points of the satellite-acquired image;

[0021] Feature matching of the multiple first feature points and the multiple second feature points is carried out to obtain multiple pairs of matching feature points.

[0022] Based on the above solution, multiple pairs of matching feature points can be obtained by feature matching of the image to be detected and the satellite-acquired image, thereby providing accurate conversion information for the perspective conversion of the image to be detected.

[0023] In a possible implementation, adjusting the water edge line of the image to be detected based on the multiple feature point pairs and the satellite-acquired image to determine the target water edge line of the image to be detected includes:

[0024] Estimating a homography matrix based on the multiple feature point pairs and a preset rule to obtain a homography matrix that conforms to the preset rule;

[0025] Performing perspective transformation on the image to be detected based on the homography matrix and the satellite-acquired image to obtain a perspective transformation image of the image to be detected; the perspective of the perspective transformation image is the same as that of the satellite-acquired image;

[0026] Adjusting the water edge line of the image to be detected based on the perspective transformation image to obtain the target water edge line of the image to be detected.

[0027] Based on the above solution, after obtaining multiple matching feature point pairs, a homography matrix that conforms to the preset rule can be obtained based on the multiple feature points, and then the perspective of the image to be detected can be transformed based on the homography matrix and the satellite-acquired image, so as to obtain a perspective transformation image of the image to be detected and a more accurate target water edge line.

[0028] In a possible implementation, sampling an image profile sequence of the video to be detected in a stacking manner based on the target water edge line of the image to be detected to obtain a time stack image of the target area includes:

[0029] Determining a profile line of the image to be detected based on the target water edge line of the image to be detected; the profile line is perpendicular to the target water edge line;

[0030] Taking the position where the profile line is located as a sampling area, and sampling the sampling area in the video to be detected in a stacking manner along the direction from the ocean to the coast according to the sampling time from early to late to obtain a time stack image of the target area within a preset sampling duration;

[0031] The horizontal axis of the time stack image represents each sampling point in the sampling area; the vertical axis of the time stack image represents the sampling time of the sampling area.

[0032] Based on the above solution, sampling the sampling area in the video to be detected in a stacking manner continuously along the direction from the ocean to the coast according to the sampling time from early to late can obtain a time stack image of the target area, providing a basis for extracting wave information.

[0033] In a possible implementation, the wave information includes wave runup; then extracting the wave information in the video to be detected in the target area based on the time-stack image includes:

[0034] Based on the time-stack image, extract the water edge contour line of the time-stack image;

[0035] Based on the water edge contour line of the time-stack image, determine each wave crest within the preset sampling duration and the corresponding relationship between each wave crest and each sampling point;

[0036] According to the pre-calibrated corresponding relationship between each sampling point and height, and the corresponding relationship between each wave crest and each sampling point, determine the height of each wave crest;

[0037] Based on the height of each wave crest, determine the wave runup within the preset sampling duration.

[0038] Based on the above solution, by obtaining the water edge contour line in the time-stack image, and then according to the wave crest information determined by the water edge contour line and the pre-calibrated corresponding relationship between each sampling point and height, the wave runup in the video to be detected in the target area can be accurately extracted.

[0039] In a possible implementation, the wave information further includes wave count and wave speed;

[0040] Then extracting the wave information in the video to be detected in the target area based on the time-stack image further includes:

[0041] Perform feature extraction and feature processing on the pre-configured text information, pre-configured visual cue information, and the time-stack image to obtain at least one detection box of the time-stack image; each detection box includes at least one wave;

[0042] Use the SAM model to perform image segmentation on the at least one detection box to obtain the wave segmentation result of the target area;

[0043] Based on the wave segmentation result and the sampling time of the sampling area, determine the wave count and the wave speed.

[0044] Based on the above solution, by performing feature extraction and image segmentation on the time-stack image, and combining the time-stack image, the wave count and the wave speed can be extracted more simply, efficiently and accurately.

[0045] In a possible implementation, performing feature extraction and feature processing on the pre-configured text information, pre-configured visual cue information, and the time-stack image to obtain at least one detection box of the time-stack image includes:

[0046] Extract features from the pre-configured text information to obtain text features;

[0047] Extract features from the pre-configured visual cue information to obtain visual cue features;

[0048] Extract features from the time-stack image to obtain visual features of the time-stack image;

[0049] Fuse the text features and the visual cue features to obtain fused features;

[0050] Decode the fused features and the visual features to obtain at least one detection box of the time-stack image.

[0051] Based on the above solution, by performing multi-modal feature extraction on the pre-configured text information, visual cue information, and time-stack image, text features, visual features, and visual cue features are obtained. Then, the extracted multi-modal features are fused and decoded, and at least one detection box of the time-stack image can be predicted, making the wave information extracted in combination with the time-stack image more accurate.

[0052] In a second aspect, an embodiment of the present application provides a device for obtaining wave information, the device including:

[0053] A feature extraction module, configured to extract features from a to-be-detected image of a target area to obtain a water line of the to-be-detected image; the to-be-detected image is any frame image extracted from a to-be-detected video of the target area;

[0054] A matching module, configured to obtain a satellite-acquired image corresponding to the target area, and perform feature matching on the to-be-detected image and the satellite-acquired image to obtain a plurality of pairs of matching feature points;

[0055] An adjustment module, configured to adjust the water line of the to-be-detected image based on the plurality of pairs of feature points and the satellite-acquired image to determine a target water line of the to-be-detected image;

[0056] An image determination module, configured to perform image profile sequence sampling on the to-be-detected video in a stacking manner based on the target water line of the to-be-detected image to obtain a time-stack image of the target area;

[0057] An information extraction module, configured to extract wave information in the to-be-detected video of the target area based on the time-stack image.

[0058] In a possible implementation manner, the feature extraction module is specifically configured to:

[0059] Extract features from the image to be detected through a semantic segmentation grid to obtain a segmentation mask image of the image to be detected;

[0060] Vectorize the segmentation mask image to obtain a water body contour line of the image to be detected;

[0061] Match the image to be detected with a preset detection area, and use the water body contour line within the preset detection area as the water edge line of the image to be detected.

[0062] In a possible implementation, the matching module is specifically configured to:

[0063] Extract features from the image to be detected to obtain a plurality of first feature points of the image to be detected, and extract features from the satellite-acquired image to obtain a plurality of second feature points of the satellite-acquired image;

[0064] Perform feature matching on the plurality of first feature points and the plurality of second feature points to obtain a plurality of pairs of matching feature points.

[0065] In a possible implementation, the adjustment module is specifically configured to:

[0066] Estimate a homography matrix based on the plurality of pairs of feature points and a preset rule to obtain a homography matrix that conforms to the preset rule;

[0067] Perform perspective transformation on the image to be detected based on the homography matrix and the satellite-acquired image to obtain a perspective transformation image of the image to be detected; the perspective of the perspective transformation image is the same as that of the satellite-acquired image;

[0068] Adjust the water edge line of the image to be detected based on the perspective transformation image to obtain a target water edge line of the image to be detected.

[0069] In a possible implementation, the image determination module is specifically configured to:

[0070] Determine a profile line of the image to be detected based on the target water edge line of the image to be detected; the profile line is perpendicular to the target water edge line;

[0071] Use the position where the profile line is located as a sampling area, and along the direction from the ocean to the coast, sample the image profile sequence of the sampling area in the video to be detected in a stacked manner according to the sampling time from early to late to obtain a time stack image of the target area within a preset sampling duration;

[0072] The horizontal axis of the time stack image represents each sampling point in the sampling area; the vertical axis of the time stack image represents the sampling time of the sampling area.

[0073] In one possible implementation, the wave information includes wave runup; the information extraction module is specifically configured to:

[0074] Extract the water edge contour line of the time stack image based on the time stack image;

[0075] Based on the water edge contour line of the time stack image, determine each wave crest within the preset sampling duration and the corresponding relationship between each wave crest and each sampling point;

[0076] Determine the height of each wave crest according to the pre-calibrated corresponding relationship between each sampling point and the height and the corresponding relationship between each wave crest and each sampling point;

[0077] Based on the height of each wave crest, determine the wave runup within the preset sampling duration.

[0078] In one possible implementation, the wave information further includes wave count and wave speed; the information extraction module is further configured to:

[0079] Perform feature extraction and feature processing on the time stack image to obtain at least one detection frame of the time stack image; each detection frame includes at least one wave;

[0080] Use the SAM model to perform image segmentation on the at least one detection frame to obtain the wave segmentation result of the target area;

[0081] Based on the wave segmentation result and the sampling time of the sampling area, determine the wave count and the wave speed.

[0082] In one possible implementation, the information extraction module is specifically configured to:

[0083] Perform feature extraction on the time stack image to obtain the text feature, visual feature, and visual prompt feature of the time stack image;

[0084] Perform feature fusion and feature decoding on the text feature, the visual feature, and the visual prompt feature to obtain at least one detection frame of the time stack image.

[0085] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0086] A memory for storing program instructions;

[0087] A processor for obtaining the program instructions stored in the memory and executing the method described in the first aspect above according to the obtained program instructions.

[0088] Fourthly, an embodiment of the present application provides a computer-readable storage medium storing computer instructions, which when running on a computer, cause the computer to execute the method described in the first aspect above.

[0089] For the technical effects brought by any implementation manner in the second aspect to the fourth aspect, reference may be made to the technical effects brought by the method described in the first aspect, which will not be elaborated herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the accompanying drawings required for description in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other accompanying drawings based on these accompanying drawings without creative efforts.

[0091] Figure 1 FIG. is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0092] Figure 2 FIG. is a schematic flowchart of a method for obtaining wave information provided by an embodiment of the present application;

[0093] Figure 3 FIG. is a schematic flowchart of a method for determining a water edge line provided by an embodiment of the present application;

[0094] Figure 4 FIG. is a schematic flowchart of a method for determining a segmentation mask map provided by an embodiment of the present application;

[0095] Figure 5 FIG. is an effect diagram of a to-be-detected image provided by an embodiment of the present application;

[0096] Figure 6 FIG. is an effect diagram of a segmentation mask map provided by an embodiment of the present application;

[0097] Figure 7 FIG. is an effect diagram of the water edge line of a to-be-detected image provided by an embodiment of the present application;

[0098] Figure 8 FIG. is a schematic diagram of a plurality of pairs of matching feature points provided by an embodiment of the present application;

[0099] Figure 9 FIG. is a schematic flowchart of a method for determining a target water edge line provided by an embodiment of the present application;

[0100] Figure 10 FIG. is a schematic diagram of a satellite-acquired image provided by an embodiment of the present application;

[0101] Figure 11 Schematic diagram of the target water line of a to-be-detected image provided by an embodiment of the present application;

[0102] Figure 12 Schematic diagram of the cross-hatching of a to-be-detected image provided by an embodiment of the present application;

[0103] Figure 13 Schematic diagram of the time-stack image of a target area provided by an embodiment of the present application;

[0104] Figure 14 Schematic flowchart of a method for obtaining wave information provided by an embodiment of the present application;

[0105] Figure 15 Schematic flowchart of a method for obtaining wave information provided by an embodiment of the present application;

[0106] Figure 16 Schematic diagram of the wave segmentation result provided by an embodiment of the present application;

[0107] Figure 17 Schematic diagram of a device for obtaining wave information provided by an embodiment of the present application;

[0108] Figure 18 Schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0109] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Among them, the described embodiments are some but not all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0110] Moreover, in the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may mean A or B; "and / or" in the text is only a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B may mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present application, "a plurality of" means two or more than two.

[0111] Hereinafter, the terms "first" and "second" are for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0112] To understand the technical solutions provided by the embodiments of the present application, the following terms are first explained.

[0113] Wave count is calculated based on the wave period within the sampling duration. The wave period generally refers to the time interval between wave crests (or wave troughs) and is measured in seconds. It reflects the frequency characteristics of the ocean surface fluctuations and is one of the basic parameters for describing wave characteristics.

[0114] Wave speed, that is, the speed at which the wave travels, mainly refers to the horizontal movement speed of the wave surface particles. Wave speed is affected by factors such as water depth and wavelength and is a key parameter for evaluating wave energy transmission efficiency and predicting the time when the wave reaches the shore.

[0115] Wave run-up refers to the phenomenon that when the wave approaches the coast, the wave height increases due to the shallowing of the water depth. This process involves complex hydrodynamic effects, including wave breaking and energy conversion. The degree of wave run-up is affected by various factors such as the steepness of the seabed topography, the wave incidence angle, wave characteristics, and seabed friction.

[0116] Wave count, speed, and run-up extraction are the core technologies for coastal zone digitization and are of crucial significance for understanding the marine dynamic environment, predicting coastal erosion, evaluating seawall design, and protecting the coastal ecology.

[0117] Accurately measuring wave count, speed, and run-up is extremely important for disaster prevention and mitigation, assessing the risk of coastal erosion, designing coastal protection facilities, and planning coastal buildings. In the research on coastal zone digitization, through modern technical means such as remote sensing satellites, unmanned aerial vehicles, and camera monitoring, parameters such as wave count, speed, and run-up can be monitored and analyzed with high precision, providing a scientific basis for coastal management, environmental protection, and disaster warning.

[0118] In the related art, through the installation and calibration of cameras, the acquisition of wave run-up images, and the analysis of image data, the spatial distribution of wave run-up is obtained, and wave run-up data is automatically extracted from the video. However, for areas with unclear water-land contrast, such as coastal tidal flats and other scenarios, the recognition effect is poor, and the wave information extraction is inaccurate.

[0119] In view of this, the present application provides a method, apparatus, device and medium for obtaining wave information, so as to solve the problems of poor recognition effect in areas with low water-land contrast and inaccurate extraction of wave information in related technologies.

[0120] The inventive concept of the embodiments of the present application can be summarized as follows: in the embodiments of the present application, feature extraction is performed on a to-be-detected image of a target area to obtain the water edge line of the to-be-detected image; the to-be-detected image is any frame image extracted from a to-be-detected video of the target area; a satellite acquisition image corresponding to the target area is obtained, and feature matching is performed on the to-be-detected image and the satellite acquisition image to obtain multiple pairs of matching feature points; based on the multiple pairs of feature points and the satellite acquisition image, the water edge line of the to-be-detected image is adjusted to determine the target water edge line of the to-be-detected image; based on the target water edge line of the to-be-detected image, image profile sequence sampling is performed on the to-be-detected video in a stacking manner to obtain a time stack image of the target area; based on the time stack image, wave information in the to-be-detected video of the target area is extracted, and the wave information can be extracted more simply, efficiently and accurately by combining the time stack image.

[0121] Thus, first, more accurate target water edge lines are obtained through feature matching of the satellite acquisition image and the to-be-detected image, and then image profile sequence sampling is performed on the to-be-detected video based on the target water edge line to obtain a time stack image of the target area, and the wave information can be extracted more simply, efficiently and accurately by combining the time stack image.

[0122] After introducing the main inventive concept of the embodiments of the present application, the application scenario diagram of the method for obtaining wave information provided by the embodiments of the present application will be described below with reference to the accompanying drawings. As Figure 1 shown, it is a schematic diagram of an application scenario of a method for obtaining wave information provided by an embodiment of the present application. Figure 1 It includes a server 100 and an electronic device 200; where:

[0123] After the electronic device 200 obtains the to-be-detected video of the target area from the server 100, it randomly extracts a frame of image as the to-be-detected image, and then performs feature extraction on the to-be-detected image to obtain the water edge line of the to-be-detected image;

[0124] After that, a satellite acquisition image corresponding to the target area is collected, and feature matching is performed on the to-be-detected image and the satellite acquisition image to obtain multiple pairs of matching feature points; based on the multiple pairs of feature points and the satellite acquisition image, the water edge line of the to-be-detected image is adjusted to determine the target water edge line of the to-be-detected image;

[0125] Based on the target water edge line of the image to be detected, the image profile sequence sampling of the video to be detected is performed in a stacked manner to obtain the time stack image of the target area; based on the time stack image, the wave information in the video to be detected in the target area is extracted.

[0126] It should be noted that a frame of image can also be randomly extracted directly in the server 100 as the image to be detected, and then the features of the image to be detected are extracted to obtain the water edge line of the image to be detected; and the satellite-acquired image corresponding to the target area is obtained, and the features of the image to be detected and the satellite-acquired image are matched to obtain multiple pairs of matching feature points;

[0127] Based on multiple pairs of feature points and the satellite-acquired image, the water edge line of the image to be detected is adjusted to determine the target water edge line of the image to be detected; based on the target water edge line of the image to be detected, the image profile sequence sampling of the video to be detected is performed in a stacked manner to obtain the time stack image of the target area; based on the time stack image, the wave information in the video to be detected in the target area is extracted;

[0128] Then, the wave information in the video to be detected in the target area is sent to the electronic device 200 for display.

[0129] Among them, the server 100 and the electronic device 200 can communicate. The communication method can be to communicate using wired communication technology, for example, by connecting a network cable or a serial port cable for communication; it can also be to communicate using wireless communication technology, for example, by technologies such as Bluetooth or WIFI, and specific limitations are not made.

[0130] To further illustrate the technical solutions provided in the embodiments of the present application, the following will be described in detail in combination with the accompanying drawings and specific implementation manners. Although the embodiments of the present application provide method operation steps as shown in the following embodiments or drawings, based on routine or non-creative labor, more or fewer operation steps may be included in the method. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiments of the present application.

[0131] After introducing the application scenarios of the embodiments of the present application, the following will be described in detail in combination with the accompanying drawings and specific implementation manners. Although the embodiments of the present application provide method operation steps as shown in the following embodiments or drawings, based on routine or non-creative labor, more or fewer operation steps may be included in the method. In steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiments of the present application.

[0132] See Figure 2 , which is a schematic flowchart of a method for obtaining wave information provided in an embodiment of the present application. As Figure 2As shown, the method includes the following steps:

[0133] In step 201, feature extraction is performed on the image to be detected in the target area to obtain the water edge line of the image to be detected; the image to be detected is any frame image extracted from the video to be detected in the target area.

[0134] In a possible implementation, in the embodiments of the present application, when performing feature extraction on the image to be detected in the target area to obtain the water edge line of the image to be detected, it can be executed as Figure 3 the steps shown:

[0135] Step 301, perform feature extraction on the image to be detected through a semantic segmentation grid to obtain a segmentation mask map of the image to be detected.

[0136] Specifically, when implementing, input the image to be detected into the semantic segmentation grid, and segment the water body in the image to be detected through the semantic segmentation grid to obtain the segmentation mask map after the water body is segmented, as Figure 4 shown.

[0137] Among them, the semantic segmentation network is mainly divided into two parts: backbone and decode_head. Among them, the backbone uses the darknet network for feature extraction, and the decode_head uses refine_head for upsampling, and finally obtains a segmentation mask map with the same size as the image to be detected.

[0138] Step 302, vectorize the segmentation mask map to obtain the water body contour line of the image to be detected.

[0139] Specifically, when implementing, use the vectorization method to obtain the water body contour line in the segmentation mask map.

[0140] Step 303, match the image to be detected with a preset detection area, and use the water body contour line located within the preset detection area as the water edge line of the image to be detected.

[0141] Specifically, when implementing, based on the configured preset detection area, the water body contour line is divided into the contour line outside the detection area and the contour line inside the detection area. Among them, the contour line inside the detection area is the finally extracted water edge line of the image to be detected.

[0142] For example, Figure 5 is the image to be detected in the target area. After Figure 4 feature extraction is performed through the semantic segmentation grid shown, the segmentation mask map shown in Figure 6 is obtained.

[0143] Finally, the segmented mask image, the image to be detected, and the preset detection area are matched to obtain the image to be detected within the preset detection area, and the water body contour line within the preset detection area is used as the water boundary line of the image to be detected.

[0144] As Figure 7 shown, the left figure is the image to be detected within the preset detection area, and the right figure is the water body contour line within the preset detection area, that is, the water boundary line of the image to be detected.

[0145] In step 202, the satellite acquisition image corresponding to the target area is obtained, and feature matching is performed on the image to be detected and the satellite acquisition image to obtain multiple pairs of matching feature points.

[0146] In a possible implementation, to perform feature matching on the image to be detected and the satellite acquisition image to obtain multiple pairs of matching feature points, the following operations can be executed:

[0147] Feature extraction is performed on the image to be detected to obtain multiple first feature points of the image to be detected, and feature extraction is performed on the satellite acquisition image to obtain multiple second feature points of the satellite acquisition image;

[0148] Feature matching is performed on the multiple first feature points and the multiple second feature points to obtain multiple pairs of matching feature points.

[0149] Specifically, when implementing, a map block of the size of the target area is intercepted on the map, the satellite acquisition image of the target area collected by the satellite is obtained and matched with the image to be detected, and then multiple first feature points of the image to be detected and multiple second feature points of the satellite acquisition image are respectively extracted through the superpoint network.

[0150] Then, based on the superglue network, the extracted feature points are matched, paired feature point pairs are obtained, and some abnormal points are removed to obtain multiple pairs of matching feature points.

[0151] As Figure 8 shown, it is a schematic diagram of multiple pairs of matching feature points.

[0152] In step 203, based on the multiple pairs of feature points and the satellite acquisition image, the water boundary line of the image to be detected is adjusted to determine the target water boundary line of the image to be detected.

[0153] In a possible implementation, to adjust the water boundary line of the image to be detected based on the multiple pairs of feature points and the satellite acquisition image to determine the target water boundary line of the image to be detected, the following steps can be executed as Figure 9 shown:

[0154] Step 901, perform homography matrix estimation based on the multiple pairs of feature points and the preset rules to obtain a homography matrix that meets the preset rules.

[0155] Step 902: Based on the homography matrix and the satellite-acquired image, perform perspective transformation on the image to be detected to obtain a perspective-transformed image of the image to be detected; the perspective-transformed image has the same perspective as the satellite-acquired image.

[0156] Step 903: Based on the perspective-transformed image, adjust the water line of the image to be detected to obtain the target water line of the image to be detected.

[0157] In specific implementation, the homography matrix is estimated based on the RANSAC method + 4-point method to obtain the homography matrix with the largest number of inliers that meet the preset rules;

[0158] Based on the obtained homography matrix and the perspective of the satellite-acquired image, perform perspective transformation on the image to be detected to obtain a perspective-transformed image of the image to be detected;

[0159] Then, according to the correspondence relationship of each pixel point between the perspective-transformed image and the image to be detected, the position of the water line of the image to be detected in the perspective-transformed image can be determined, thereby obtaining the water line of the perspective-transformed image, that is, the target water line of the image to be detected.

[0160] According to business requirements, if it is necessary to determine the water line of the satellite-acquired image, then according to the correspondence relationship of each pixel point between the satellite-acquired image and the perspective-transformed image, the position of the water line of the perspective-transformed image in the satellite-acquired image can be determined, that is, the water line of the satellite-acquired image can be obtained.

[0161] Exemplarily, as Figure 10 shown, it is a schematic diagram of the satellite-acquired image. The image to be detected is converted into a perspective-transformed image with the same perspective as the satellite-acquired image according to the above method. As Figure 11 shown, the left figure is the perspective-transformed image of the image to be detected and the target water line of the image to be detected, and the right figure is the water line of the satellite-acquired image.

[0162] Among them, the left figure and the right figure have the same perspective.

[0163] In step 204, based on the target water line of the image to be detected, perform image profile sequence sampling on the video to be detected in a stacking manner to obtain a time-stack image of the target area.

[0164] In a possible implementation manner, based on the target water line of the image to be detected, performing image profile sequence sampling on the video to be detected in a stacking manner to obtain a time-stack image of the target area is executed as follows:

[0165] Based on the target water line of the image to be detected, determine the profile line of the image to be detected; the profile line is perpendicular to the target water line.

[0166] Take the position where the cross - hatch line is located as the sampling area. Along the direction from the ocean to the coast, in the order of sampling time from early to late, stack - sample the sampling area in the video to be detected to obtain a time - stack image of the target area within a preset sampling duration.

[0167] Among them, the horizontal axis of the time - stack image represents each sampling point in the sampling area; the vertical axis of the time - stack image represents the sampling time of the sampling area.

[0168] In specific implementation, first select the cross - hatch line:

[0169] Select the direction perpendicular to the target water - edge line in the image to be detected, that is, perpendicular to the shoreline, set up a sampling array. The array extends seaward beyond the wave - breaking zone and shoreward beyond the maximum run - up position of the waves to obtain the cross - hatch line of the image to be detected.

[0170] As Figure 12 shown, it is a schematic diagram of the cross - hatch line of the image to be detected. The cross - hatch line of the image to be detected is perpendicular to the target water - edge line.

[0171] Then take the position where the cross - hatch line is located as the sampling area, and stack - sample the sampling area in the video to be detected from bottom to top in the order of sampling time from early to late to obtain a time - stack image of the target area within a preset sampling duration. The sampling direction is along the direction from the ocean to the coast.

[0172] As Figure 13 shown, it is a schematic diagram of the time - stack image of the target area. The horizontal axis represents each sampling point in the sampling area, and the vertical axis represents the sampling time of the sampling area.

[0173] For example, Figure 12 if there are 100 sampling points on the cross - hatch line (sampling area) shown, then the horizontal axis of the time - stack image starts from the origin and successively represents the 0th to 100th sampling points, and the origin is the sampling point at the farthest distance from the shore on the cross - hatch line. If the preset sampling duration is 200 seconds, then the vertical axis of the time - stack image starts from the origin and successively represents the 0th to 200th seconds.

[0174] In step 205, based on the time - stack image, extract the wave information in the video to be detected in the target area.

[0175] In the embodiments of the present application, the wave information includes wave run - up, wave count, wave speed and other information.

[0176] If the wave information is wave run - up, then in the embodiments of the present application, based on the time - stack image, to extract the wave information in the video to be detected in the target area, the following steps can be executed as Figure 14 shown:

[0177] Step 1401: Extract the water edge contour line of the time-stack image based on the time-stack image.

[0178] Step 1402: Based on the water edge contour line of the time-stack image, determine each wave crest within the preset sampling duration and the corresponding relationship between each wave crest and each sampling point.

[0179] Step 1403: According to the pre-calibrated corresponding relationship between each sampling point and height, and the corresponding relationship between each wave crest and each sampling point, determine the height of each wave crest.

[0180] Step 1404: Based on the height of each wave crest, determine the wave run-up within the preset sampling duration.

[0181] In specific implementation, first, the water edge contour line of the time-stack image can be extracted according to the time-stack image. Then, each wave crest and the sampling point to which each wave crest belongs can be identified based on the water edge contour line.

[0182] The height represented by each sampling point can be pre-calibrated. That is, information such as the actual distance and actual height represented by each sampling point can be marked by dotting. Then, according to the proportional relationship between the time-stack image and the actual measurement information, the wave run-up within the preset sampling duration can be determined.

[0183] Specifically, information such as the maximum wave run-up, minimum wave run-up, average offshore distance, and wave run-up in the target area within the preset sampling duration can be statistically analyzed according to business needs.

[0184] For example, first, calibrate the heights corresponding to 100 sampling points. Then, according to the corresponding relationship between each wave crest and each sampling point, the height corresponding to each wave crest can be determined. Furthermore, according to the proportional relationship between the time-stack image and the actual measurement information, the actual height corresponding to each wave crest can be determined.

[0185] At this time, if it is necessary to statistically analyze the maximum wave run-up and minimum wave run-up in the target area within the preset sampling duration, by comparing the actual heights corresponding to each wave crest, the maximum wave run-up and minimum wave run-up can be directly selected.

[0186] If it is necessary to statistically analyze the average wave run-up in the target area within the preset sampling duration, the actual heights corresponding to all wave crests within the preset sampling duration can be added up and then divided by the number of wave crests to obtain the average wave run-up within the preset sampling duration.

[0187] Based on the time-stack image, the water edge contour line can be determined by the following several methods:

[0188] Method 1: Obtain the water edge contour line of the time-stack image by performing semantic segmentation and vectorization on the time-stack image.

[0189] Method 2: Connect the intersection points of the target water edge line and the profile line of each frame image in the video to be detected to obtain the water edge contour line of the time stack image.

[0190] Method 3: Perform image segmentation on the time stack image to obtain the wave segmentation result of the target area; determine each wave in the time stack image based on the wave segmentation result, and connect the highest points of each wave to obtain the water edge contour line of the time stack image.

[0191] After determining the water edge contour line of the time stack image, each wave crest on the water edge contour line can be determined, and then each sampling point corresponding to each wave crest can be determined.

[0192] If the wave information is wave count and wave speed, in the embodiments of the present application, based on the time stack image, extracting the wave information in the video to be detected in the target area can be performed as Figure 15 shown in the steps:

[0193] Step 1501, perform feature extraction and feature processing on the pre-configured text information, the pre-configured visual prompt information, and the time stack image to obtain at least one detection frame of the time stack image; each detection frame includes at least one wave.

[0194] In a possible implementation, performing feature extraction and feature processing on the pre-configured text information, the pre-configured visual prompt information, and the time stack image to obtain at least one detection frame of the time stack image can be performed as:

[0195] Perform feature extraction on the pre-configured text information to obtain text features;

[0196] Perform feature extraction on the pre-configured visual prompt information to obtain visual prompt features;

[0197] Perform feature extraction on the time stack image to obtain the visual features of the time stack image;

[0198] Perform feature fusion on the text features and the visual prompt features to obtain fusion features;

[0199] Perform feature decoding on the fusion features and the visual features to obtain at least one detection frame of the time stack image.

[0200] Among them, the pre-configured text information can be set according to actual business requirements. For example, in the embodiments of the present application, if wave information needs to be extracted, the pre-configured text information can be text information related to waves such as wave detection and extracting wave contours, that is, the pre-configured text information gives text prompts related to waves.

[0201] Among them, the pre-configured visual prompt information can be set according to actual business requirements. For example, if wave information needs to be extracted in the embodiments of this application, the pre-configured visual prompt information can be a detection box including a wave image, or a rotation box including a wave image, that is, a visual prompt of the wave is given within the detection box (rotation box).

[0202] In specific implementation, the model can be trained first using text information, visual information (time stack images), and visual prompt information (point information (Point) and box information (Box)), and then the trained model is used to process the time stack images.

[0203] First, the feature extraction module is used to extract features from the pre-configured text information, the pre-configured visual prompt information, and the time stack images, to obtain text features corresponding to the pre-configured text information, visual prompt features corresponding to the pre-configured visual prompt information, and visual features of the time stack images.

[0204] The feature extraction module includes a text feature extraction unit, a visual feature extraction unit, and a visual prompt (Prompt) position encoding unit:

[0205] The text feature extraction unit encodes the pre-configured text information using the CLIP network to obtain text features;

[0206] The visual feature extraction unit uses the Swin-Transformer network to perform feature encoding on the time stack images to obtain visual features;

[0207] The visual prompt (Prompt) position encoding unit encodes the pre-configured visual prompt information to obtain visual prompt features. That is, this unit can encode point information (Point) and box information (Box): given normalized image point coordinates or box coordinates, they are encoded into position embeddings (Position Embedding) using the sine-cosine encoding method, to realize the conversion of interactive visual prompts from the image coordinate space to the image feature space, and obtain visual prompt features.

[0208] In specific implementation, first, K normalized point coordinates p i or normalized rotation box coordinates b i are specified:

[0209] p i =(x i ,y i ), i ∈ {1, 2,..., K}

[0210] b i =(x i ,y i ,w i ,hi , α i ), where \(i\in\{1, 2, \ldots, K\}\)

[0211] The normalized point coordinates \(p\) are encoded into the position embedding \(P\) using the sine-cosine encoding method, or the normalized rotated box coordinates \(b\) i are encoded into the position embedding \(B\): i

[0212] \(P = \text{Linear}(PE(p_1, \ldots, p\) K ); \theta P )

[0213] \(B = \text{Linear}(PE(b_1, \ldots, b\) K ) ; \theta B )

[0214] where \(PE\) represents the position embedding, and \(\text{Linear}(\cdot; \theta)\) represents a learnable linear projection layer with parameter \(\theta\). In addition to the point and rotated box cues, a learnable content embedding can be initialized, which is broadcast \(K\) times.

[0215] Furthermore, according to business requirements, a general class token \(C'\) can also be used to aggregate features from other visual cues to adapt to the situation where the user provides multiple visual cues in a single image.

[0216] These content embeddings are concatenated with the position embeddings in the channel dimension and a linear layer is applied for projection to construct the input query embedding \((Q)\):

[0217]

[0218] where \(\text{CAT}\) represents concatenation in the channel dimension. \(B'\) and \(P'\) represent global position embeddings, which are derived from the global normalized coordinates \([0.5, 0.5, 1, 1]\) and \([0.5, 0.5]\). The purpose of the global query is to aggregate features from other queries.

[0219] Subsequently, the multi-scale deformable cross-attention layer is used to extract visual cue features from the multi-scale feature maps, conditioned on the visual cue information. For the \(j\)-th visual cue information, the query feature \(Q\) after cross-attention j ′ is calculated as follows:

[0220]

[0221] The deformable attention is conditioned on the coordinates of the visual cue, i.e., each query will selectively attend to a limited set of multi-scale image features that contain the region around the visual cue. This ensures that the visual cue features representing the object of interest are captured. ​

[0222] After the extraction process, a self-attention layer is used to adjust the relationship between different queries, and a feed-forward layer is used for projection. The output of the global content query will be used as the final visual cue feature V:

[0223] V = FFN(SelfAttn(Q ′ )[-1]

[0224] Thus, by extracting features from the pre-configured visual cue information (wave-related point information or rotated box information), the visual cue feature can be obtained.

[0225] Then, a cross-modal feature fusion module is used to fuse the text features and visual cue features extracted above to obtain the fused features.

[0226] To integrate text features and visual cue features in the model, regional contrastive learning is adopted in the embodiments of this application to align these modalities.

[0227] Specifically, given the input temporal stack image and the (K) visual cue features V = (v1, …, v K ) extracted from the visual cue position encoding unit, and the text features T = (t1, …, t K ) of each cue region, the InfoNCE loss between the two types of features is calculated:

[0228]

[0229] Finally, the feature fusion of text features and visual cue features is realized to obtain the fused features.

[0230] Then, a cross-modal feature decoding module is used to perform feature decoding on the fused features and visual features using a decoder similar to DETR to predict the detection boxes, and at least one detection box of the temporal stack image is obtained. Each query is represented as a 4D anchor coordinate and is iteratively optimized between decoder layers. The query selection layer proposed in Grounding DINO is used to initialize the anchor coordinates (x, y, w, h).

[0231] Specifically, first, the similarity between the visual features and the fused features obtained by the cross-modal feature fusion module is calculated, and multiple indices with higher similarity are selected to initialize the position encoding. For example, the first 900 indices with higher similarity can be selected to initialize the position encoding.

[0232] Subsequently, the detection query focuses on the encoded multi-scale image features using deformable cross-attention and predicts the anchor offsets (Δx, Δy, Δw, Δh) at each decoder layer. The finally predicted detection box is obtained by adding the anchor and the offset:

[0233] (Δx, Δy, Δw, Δh) = MLP(Q dec )

[0234] Box = (x + Δx, y + Δy, w + Δw, h + Δh)

[0235] where Q dec is the query predicted from the box decoder. The prediction of the class uses the visual cue features as the weights of the classification layer, and the implementation is as follows:

[0236]

[0237] Thus, the cross-modal feature decoding module is used to decode the visual features and the fusion features to obtain at least one detection box of the time-stack image.

[0238] At this time, only at least one detection box of the time-stack image is obtained, and the wave form represented in the at least one detection box is not obtained. Therefore, in the embodiment of the present application, step 1502 needs to be further executed, and the SAM model is used to perform image segmentation on the at least one detection box to obtain the wave segmentation result of the target area.

[0239] Specifically in implementation, the at least one detection box output by the cross-modal feature decoding module is sequentially input into the SAM large model to obtain the wave form included in each detection box, that is, the wave segmentation result of the target area. As Figure 16 shown, it is the wave segmentation result of the time-stack image.

[0240] Step 1503, based on the wave segmentation result and the sampling time of the sampling area, determine the wave count and the wave speed.

[0241] After the wave segmentation result of the time-stack image is extracted, the wave contour line is further extracted from the wave segmentation result; then, according to the wave contour line and the sampling time, the wave count and the wave speed can be obtained.

[0242] Specifically in implementation, according to the texture difference between the surge, the wave and the sea water surface, wave detection can be performed according to the wave contour line to obtain the wave count. At the same time, according to the wave count and the cumulative sampling duration of the vertical axis, the average wave period within the cumulative sampling duration can be obtained.

[0243] According to the wave contour line extracted from the wave segmentation result and the sampling time of the time-stack image, the wave speed can be calculated.

[0244] It should be noted that the wave velocity is the instantaneous / average wave velocity in the direction of the profile line. If it is necessary to convert it into the actual wave velocity, it is necessary to further convert it into the actual wave velocity according to the calibration information of the profile line and the map.

[0245] Finally, the reporting frequency of the wave information can also be determined according to the actual business requirements.

[0246] Based on the foregoing description, the embodiment of the present application extracts features from the image to be detected in the target area to obtain the water edge line of the image to be detected; the image to be detected is any frame image extracted from the video to be detected in the target area; the satellite acquisition image corresponding to the target area is obtained, and feature matching is performed on the image to be detected and the satellite acquisition image to obtain multiple pairs of matching feature points; based on the multiple pairs of feature points and the satellite acquisition image, the water edge line of the image to be detected is adjusted to determine the target water edge line of the image to be detected; based on the target water edge line of the image to be detected, the image profile sequence sampling is performed on the video to be detected in a stacking manner to obtain the time stack image of the target area; based on the time stack image, the wave information in the video to be detected in the target area is extracted.

[0247] Thus, through the feature matching of the satellite acquisition image and the image to be detected, a more accurate target water edge line is obtained, and then the image profile sequence sampling is performed on the video to be detected based on the target water edge line to obtain the time stack image of the target area. Combining the time stack image can extract the wave information more simply, efficiently and accurately.

[0248] Based on the same inventive concept, the embodiment of the present application also provides an apparatus for obtaining wave information. As Figure 17 shown, the apparatus includes a feature extraction module 1701, a matching module 1702, an adjustment module 1703, an image determination module 1704 and an information extraction module 1705, wherein:

[0249] The feature extraction module 1701 is configured to extract features from the image to be detected in the target area to obtain the water edge line of the image to be detected; the image to be detected is any frame image extracted from the video to be detected in the target area;

[0250] The matching module 1702 is configured to obtain the satellite acquisition image corresponding to the target area, and perform feature matching on the image to be detected and the satellite acquisition image to obtain multiple pairs of matching feature points;

[0251] The adjustment module 1703 is configured to adjust the water edge line of the image to be detected based on the multiple pairs of feature points and the satellite acquisition image to determine the target water edge line of the image to be detected;

[0252] An image determination module 1704, configured to perform image profile sequence sampling on the to-be-detected video in a stacking manner based on the target water edge line of the to-be-detected image, so as to obtain a time-stack image of the target area;

[0253] An information extraction module 1705, configured to extract wave information in the to-be-detected video of the target area based on the time-stack image.

[0254] In a possible implementation manner, the feature extraction module 1701 is specifically configured to:

[0255] Extract features from the to-be-detected image through a semantic segmentation grid to obtain a segmentation mask image of the to-be-detected image;

[0256] Vectorize the segmentation mask image to obtain a water body contour line of the to-be-detected image;

[0257] Match the to-be-detected image with a preset detection area, and use the water body contour line located within the preset detection area as the water edge line of the to-be-detected image.

[0258] In a possible implementation manner, the matching module 1702 is specifically configured to:

[0259] Extract features from the to-be-detected image to obtain a plurality of first feature points of the to-be-detected image, and extract features from the satellite-acquired image to obtain a plurality of second feature points of the satellite-acquired image;

[0260] Perform feature matching on the plurality of first feature points and the plurality of second feature points to obtain a plurality of pairs of matching feature points.

[0261] In a possible implementation manner, the adjustment module 1703 is specifically configured to:

[0262] Estimate a homography matrix based on the plurality of pairs of feature points and a preset rule to obtain a homography matrix that conforms to the preset rule;

[0263] Perform perspective transformation on the to-be-detected image based on the homography matrix and the satellite-acquired image to obtain a perspective transformation image of the to-be-detected image; the perspective transformation image has the same perspective as the satellite-acquired image;

[0264] Adjust the water edge line of the to-be-detected image based on the perspective transformation image to obtain the target water edge line of the to-be-detected image.

[0265] In a possible implementation manner, the image determination module 1704 is specifically configured to:

[0266] Determine the profile line of the image to be detected based on the target water edge line of the image to be detected; the profile line is perpendicular to the target water edge line;

[0267] Take the position where the profile line is located as the sampling area, and along the direction from the ocean to the coast, sample the image profile sequence of the sampling area in the video to be detected in a stacked manner according to the sampling time from early to late, so as to obtain the time stack image of the target area within the preset sampling duration;

[0268] The horizontal axis of the time stack image represents each sampling point in the sampling area; the vertical axis of the time stack image represents the sampling time of the sampling area.

[0269] In a possible implementation, the wave information includes wave runup; the information extraction module 1705 is specifically configured to:

[0270] Extract the water edge contour line of the time stack image based on the time stack image;

[0271] Based on the water edge contour line of the time stack image, determine each wave crest within the preset sampling duration and the corresponding relationship between each wave crest and each sampling point;

[0272] According to the pre-calibrated corresponding relationship between each sampling point and height, and the corresponding relationship between each wave crest and each sampling point, determine the height of each wave crest;

[0273] Based on the height of each wave crest, determine the wave runup within the preset sampling duration.

[0274] In a possible implementation, the wave information further includes wave count and wave speed; the information extraction module 1705 is further configured to:

[0275] Perform feature extraction and feature processing on the pre-configured text information, pre-configured visual cue information, and the time stack image to obtain at least one detection box of the time stack image; each detection box includes at least one wave;

[0276] Use the SAM model to perform image segmentation on the at least one detection box to obtain the wave segmentation result of the target area;

[0277] Based on the wave segmentation result and the sampling time of the sampling area, determine the wave count and the wave speed.

[0278] In a possible implementation, the information extraction module 1705 is specifically configured to:

[0279] Extract features from the pre-configured text information to obtain text features;

[0280] Extract features from the pre-configured visual cue information to obtain visual cue features;

[0281] Extract features from the time-stack image to obtain the visual features of the time-stack image;

[0282] Fuse the text features and the visual cue features to obtain fused features;

[0283] Decode the fused features and the visual features to obtain at least one detection box of the time-stack image.

[0284] Based on the same inventive concept, an embodiment of the present application also provides an electronic device 1800. The following will refer to Figure 18 to describe the electronic device 1800 according to this embodiment of the present application. Figure 18 The illustrated electronic device 1800 is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0285] As Figure 18 shown, the electronic device 1800 is presented in the form of a general-purpose electronic device. The components of the electronic device 1800 may include, but are not limited to: the above-mentioned at least one processor 1801, the above-mentioned at least one memory 1802, and a bus 1803 connecting different system components (including the memory 1802 and the processor 1801).

[0286] The bus 1803 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a processor, or a local bus using any bus structure in a variety of bus structures.

[0287] The memory 1802 may include a readable medium in the form of volatile memory, such as random access memory (RAM) and / or cache memory, and may further include read-only memory (ROM).

[0288] The memory 1802 may also include a program / utility having a set (at least one) of program modules, and such program modules include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples.

[0289] The electronic device 1800 can also communicate with one or more external devices 1804 (such as a keyboard, a pointing device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 1800, and / or communicate with any device (such as a router, a modem, etc.) that enables the electronic device 1800 to communicate with one or more other electronic devices. Such communication can be carried out through the input / output (I / O) interface 1805. Moreover, the electronic device 1800 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 1806. As shown in the figure, the network adapter 1806 communicates with other modules for the electronic device 1800 through the bus 1803. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 1800, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0290] In an exemplary embodiment, there is also provided a computer-readable storage medium including instructions, such as the memory 1802 including instructions, and the above instructions can be executed by the processor 1801 to complete the above method for obtaining wave information. Optionally, the storage medium can be a non-transitory computer-readable storage medium. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0291] In an exemplary embodiment, there is also provided a computer program product including a computer program, and when the computer program is executed by the processor 1801, it implements any of the methods for obtaining wave information provided in this application.

[0292] In an exemplary embodiment, various aspects of a method for obtaining wave information provided in this application can also be implemented in the form of a program product, which includes program code. When the program product runs on a computer device, the program code is used to cause the computer device to execute the steps in the method for obtaining wave information according to various exemplary embodiments described above in this specification.

[0293] The program product may employ any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0294] The program product of the method for obtaining wave information according to an embodiment of the present application may employ a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on an electronic device. However, the program product of the present application is not limited thereto. In this document, the readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0295] The readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal may take various forms, including - but not limited to - an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0296] The program code contained on the readable medium may be transmitted using any appropriate medium, including - but not limited to - wireless, wired, optical fiber, RF, etc., or any suitable combination of the foregoing.

[0297] The program code for performing the operations of this application can be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's electronic device, partially on the user's device, executed as an independent software package, partially on the user's electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In the case of a remote electronic device, the remote electronic device can be connected to the user's electronic device through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external electronic device (for example, by using an Internet service provider to connect through the Internet).

[0298] It should be noted that although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more of the above-described units can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0299] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

[0300] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0301] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable image scaling devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable image scaling devices generate means for implementing the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0302] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable image scaling device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0303] These computer program instructions can also be loaded onto a computer or other programmable image scaling device, such that a series of operational steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0304] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present application.

[0305] Obviously, those skilled in the art can make various changes and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A method for obtaining wave information, characterized in that The method includes: Performing feature extraction on the image to be detected in the target area to obtain the water edge line of the image to be detected; the image to be detected is any frame image extracted from the video to be detected in the target area; Obtaining the satellite-acquired image corresponding to the target area, and performing feature matching on the image to be detected and the satellite-acquired image to obtain multiple pairs of matching feature points; Based on the multiple pairs of feature points and the satellite-acquired image, adjusting the water edge line of the image to be detected to determine the target water edge line of the image to be detected; Based on the target water edge line of the image to be detected, performing image profile sequence sampling on the video to be detected in a stacking manner to obtain the time stack image of the target area; Based on the time stack image, extracting the wave information in the video to be detected in the target area.

2. The method according to claim 1, wherein The performing feature extraction on the image to be detected in the target area to obtain the water edge line of the image to be detected includes: Performing feature extraction on the image to be detected through a semantic segmentation grid to obtain the segmentation mask image of the image to be detected; Vectorizing the segmentation mask image to obtain the water body contour line of the image to be detected; Matching the image to be detected with a preset detection area, and taking the water body contour line located within the preset detection area as the water edge line of the image to be detected.

3. The method according to claim 1, wherein The adjusting the water edge line of the image to be detected based on the multiple pairs of feature points and the satellite-acquired image to determine the target water edge line of the image to be detected includes: Estimating a homography matrix based on the multiple pairs of feature points and a preset rule to obtain a homography matrix that conforms to the preset rule; Based on the homography matrix and the satellite-acquired image, performing perspective transformation on the image to be detected to obtain the perspective transformation image of the image to be detected; the perspective transformation image and the satellite-acquired image have the same perspective; Adjusting the water edge line of the image to be detected based on the perspective transformation image to obtain the target water edge line of the image to be detected.

4. The method according to claim 1, characterized in that, The performing image profile sequence sampling on the video to be detected in a stacking manner based on the target water edge line of the image to be detected to obtain the time stack image of the target area includes: Based on the target water edge line of the image to be detected, determining the profile line of the image to be detected; the profile line is perpendicular to the target water edge line; Taking the position where the profile line is located as the sampling area, and performing image profile sequence sampling on the sampling area in the video to be detected in a stacking manner along the direction from the ocean to the coast according to the sampling time from early to late to obtain the time stack image of the target area within a preset sampling duration; The horizontal axis of the time stack image represents each sampling point in the sampling area; the vertical axis of the time stack image represents the sampling time of the sampling area.

5. The method according to claim 4, characterized in that The wave information includes wave runup; then the extracting the wave information in the video to be detected in the target area based on the time stack image includes: Based on the time stack image, extracting the water edge contour line of the time stack image; Based on the water edge contour line of the time stack image, determine each wave crest within the preset sampling duration and the corresponding relationship between each wave crest and each sampling point; According to the pre-calibrated corresponding relationship between each sampling point and height, and the corresponding relationship between each wave crest and each sampling point, determine the height of each wave crest; Based on the height of each wave crest, determine the wave run-up within the preset sampling duration.

6. The method according to claim 5, wherein The wave information further includes wave count and wave speed; Then, the extracting of the wave information in the to-be-detected video of the target area based on the time stack image further includes: Performing feature extraction and feature processing on the pre-configured text information, visual prompt information, and the time stack image to obtain at least one detection frame of the time stack image; each detection frame includes at least one wave; Using the SAM model to perform image segmentation on the at least one detection frame to obtain the wave segmentation result of the target area; Based on the wave segmentation result and the sampling time of the sampling area, determine the wave count and the wave speed.

7. The method according to claim 6, characterized in that, The performing feature extraction and feature processing on the pre-configured text information, pre-configured visual prompt information, and the time stack image to obtain at least one detection frame of the time stack image includes: Performing feature extraction on the pre-configured text information to obtain text features; Performing feature extraction on the pre-configured visual prompt information to obtain visual prompt features; Performing feature extraction on the time stack image to obtain the visual features of the time stack image; Performing feature fusion on the text features and the visual prompt features to obtain fusion features; Performing feature decoding on the fusion features and the visual features to obtain at least one detection frame of the time stack image.

8. An apparatus for obtaining wave information, characterized in that, The device includes: A feature extraction module, configured to perform feature extraction on the to-be-detected image of the target area to obtain the water edge line of the to-be-detected image; the to-be-detected image is any frame image extracted from the to-be-detected video of the target area; A matching module, configured to obtain the satellite acquisition image corresponding to the target area, and perform feature matching on the to-be-detected image and the satellite acquisition image to obtain multiple pairs of matching feature points; An adjustment module, configured to adjust the water edge line of the to-be-detected image based on the multiple pairs of feature points and the satellite acquisition image to determine the target water edge line of the to-be-detected image; An image determination module, configured to perform image profile sequence sampling on the to-be-detected video in a stacking manner based on the target water edge line of the to-be-detected image to obtain the time stack image of the target area; An information extraction module, configured to extract the wave information in the to-be-detected video of the target area based on the time stack image.

9. An electronic device, characterized in that, Includes: A memory, configured to store program instructions; A processor, configured to obtain the program instructions stored in the memory and execute the method according to any one of claims 1-7 according to the obtained program instructions.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when run on a computer, cause the computer to execute the method according to any one of claims 1-7.

Images