A measurement method, apparatus, electronic device, and storage medium

Abstract

This application provides a measurement method, apparatus, electronic device, and storage medium. The measurement method includes: acquiring a target video file corresponding to a river segment to be measured within a target time period, the target video file including video images of the river segment to be measured on a target plane, the target plane being parallel to the horizontal end face of the river segment to be measured; processing the target video images based on a first semantic segmentation model to obtain a target mask image, the target video image being any frame of video image in the target video file, the mask portion corresponding to the target mask image including bank ice and ice floes; performing ice floe recognition on each frame of video image in the target video file to obtain a set of target ice floe trajectory points; and determining the open river width of the river segment to be measured based on the target video image, the target mask image, and the set of target ice floe trajectory points. This application achieves the goal of measuring the open river width by processing video files, reducing the degree of human intervention in the measurement process and improving the accuracy of the open river width corresponding to the river segment to be measured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video image processing, in particular to a measurement method and device, electronic equipment and storage medium. BACKGROUND

[0002] At present, the common open river width measurement during ice period mainly uses observation instruments to observe the river surface in sections, and determines the open river width proportion by manual reading, and finally obtains the total open river width. The whole measurement process needs manual participation, which is laborious and prone to errors, and is not conducive to the development of ice flood prevention work. SUMMARY

[0003] The embodiments of the present application provide a measurement method and device, electronic equipment and storage medium to solve the problem of how to measure the open river width during the ice period.

[0004] In a first aspect, the embodiments of the present application provide a measurement method, which comprises:

[0005] obtaining a target video file corresponding to a to-be-measured river section within a target time length, the target video file comprising video images corresponding to a target plane of the to-be-measured river section, and the target plane being parallel to the horizontal end surface of the to-be-measured river section;

[0006] processing the target video images based on a first semantic segmentation model to obtain a target mask image corresponding to the target video images, the target video images being any frame video images in the target video file, and the mask part corresponding to the target mask image comprising bank ice and ice;

[0007] performing ice identification on each frame video image in the target video file to obtain a target ice trajectory point set;

[0008] determining an open river width corresponding to the to-be-measured river section according to the target video images, the target mask image and the target ice trajectory point set.

[0009] In a second aspect, the embodiments of the present application further provide a measurement device, which comprises:

[0010] a first obtaining module configured to obtain a target video file corresponding to a to-be-measured river section within a target time length, the target video file comprising video images corresponding to a target plane of the to-be-measured river section, and the target plane being parallel to the horizontal end surface of the to-be-measured river section;

[0011] a second obtaining module configured to process the target video images based on a first semantic segmentation model to obtain a target mask image corresponding to the target video images, the target video images being any frame video images in the target video file, and the mask part corresponding to the target mask image comprising bank ice and ice;

[0012] a third obtaining module, configured to perform ice identification on each frame of video image in the target video file to obtain a target ice trajectory point set;

[0013] a first determining module, configured to determine the open river width corresponding to the to-be-measured river section according to the target video image, the target mask image and the target ice trajectory point set.

[0014] In a third aspect, an electronic device is provided, which includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the measurement method described above is implemented.

[0015] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the measurement method described above is implemented.

[0016] The embodiments of the present application at least have the following technical effects:

[0017] The technical scheme of the embodiments of the present application obtains the target video image, the target mask image including the bank ice and the ice trajectory point set by processing the target video file corresponding to the to-be-measured river section, and then determines the open river width of the to-be-measured river section, thereby achieving the purpose of measuring the open river width by processing the video file, reducing the participation of artificial in the measurement process, and further improving the accuracy of the open river width corresponding to the to-be-measured river section. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical schemes of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description.

[0019] Figure 1 is a flowchart of the measurement method provided by the embodiments of the present application;

[0020] Figure 2 is a structural schematic diagram of the measurement device provided by the embodiments of the present application;

[0021] Figure 3 is a block diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0022] With reference to the drawings and the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the scope of the present application.

[0023] It should be understood that every technical feature mentioned in the specification refers to a specific feature of the embodiments, which is included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner.

[0024] In various embodiments of the present application, it should be understood that the size of the serial number of the following processes does not mean the order of execution, and the execution order of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0025] As shown in Figure 1 The present application provides a measurement method, which comprises:

[0026] In step 101, a target video file corresponding to a to-be-measured river section in a target time length is obtained, the target video file comprising a video image corresponding to a target plane of the to-be-measured river section, and the target plane being parallel to a horizontal end surface of the to-be-measured river section.

[0027] In order to obtain the open river width of the to-be-measured river section during icing, a target video file can be obtained by photographing the to-be-measured river section through a camera, and then the open river width of the to-be-measured river section can be obtained by analyzing and processing the target video file. The video image in the target video file corresponds to a target plane parallel to the horizontal end surface of the to-be-measured river section. The target video file can be a video file directly obtained by photographing the to-be-measured river section from directly above at a certain height from the to-be-measured river section and at a front photographing angle, or can be a source video file obtained by photographing the to-be-measured river section at a certain height from the ground near the to-be-measured river section and at a slanting side photographing angle, and then the source video file is changed in perspective to convert the video image corresponding thereto to the target plane to obtain the target video file.

[0028] Since the ice floes floating in the river during the ice period will move with the water flow, in order to prevent the ice floes from being mistaken for bank ice, causing the final open river width to be inaccurate, it is necessary to observe the to-be-measured river section for a period of time, that is, to obtain a video file of a target time length. Specifically, the length of the target time length needs to be determined according to the flow velocity and the density of the ice floes in the river section, and the present application does not make specific limitations.

[0029] In step 102, the target video image is processed based on the first semantic segmentation model to obtain a target mask corresponding to the target video image, the target video image being any frame video image in the target video file, and the mask part corresponding to the target mask including bank ice and ice floes.

[0030] The semantic segmentation model is used to classify each pixel point of an image, and the image is segmented into a plurality of visually meaningful or interesting regions. The first semantic segmentation model here can segment the ice surface in the image as an interesting region through pre-training, so that the mask part corresponding to the target mask obtained after the target video image is processed based on the first semantic segmentation model is the ice surface, specifically including bank ice and ice floes, that is, the ice surface part in the target video image can be segmented out by the first semantic segmentation model. It should be noted that the mask part here includes bank ice and ice floes, since the target mask is obtained based on a static target video image, it cannot be determined whether the ice surface at the edge position is bank ice or ice floes.

[0031] Specifically, the ice floes in the river will move with the water flow, so the positions of the ice floes in each frame video image in the target video file are different from each other, and the target video image here is used as a basic image in the subsequent processing process, specifically in determining the open river width, the target mask and the ice floe trajectory set need to be combined, therefore, as long as the basic image includes the bank ice and ice floes of the to-be-measured river section, any frame video image in the target video file can be selected as the target video image.

[0032] In step 103, ice floe recognition is performed on each frame video image in the target video file to obtain a target ice floe trajectory point set.

[0033] Specifically, when there are ice floes floating on the river surface and moving with the river in the to-be-measured river section, in order to obtain the open river width of the to-be-measured river section, after determining the target video image of the to-be-measured river section and the target mask corresponding to the target video image, the trajectory point set of the ice floes appearing in the to-be-measured river section also needs to be determined.

[0034] In step 104, the open river width corresponding to the to-be-measured river section is determined according to the target video image, the target mask, and the target ice floe trajectory point set.

[0035] After acquiring the target video image, the target mask image, and the target ice floe trajectory point set corresponding to the river section to be tested, the target video image is used as the base image. By combining the target mask image and the target ice floe trajectory point set, it can be determined whether the ice surface connected to the flowing river in the target video image is shore ice or ice floes. This allows us to determine the distance between the shore ice on both sides of the river section to be tested, which is the open river width corresponding to the river section to be tested. Here, the open river width refers to the minimum distance between the shore ice in the river width direction in the river section to be tested.

[0036] In this embodiment, the target video file corresponding to the river section to be measured is processed to obtain the target video image, the target mask image including the bank ice and ice floes, and the target ice floe trajectory point set, thereby determining the exposed river width of the river section to be measured. This achieves the purpose of measuring the exposed river width by processing the video file, reduces the degree of human involvement in the measurement process, and thus improves the accuracy of the exposed river width corresponding to the river section to be measured.

[0037] In one optional embodiment of this application, step 101, obtaining the target video file corresponding to the river segment to be tested within the target duration, includes one of the following schemes:

[0038] Acquire the target video file, which is taken directly above the river section to be measured and corresponds to the target duration;

[0039] The target video file is obtained by performing perspective transformation on the first video file. The first video file is a video file corresponding to the target duration obtained by shooting the river section to be tested at a preset position. The preset position is a position located on one side of the river section to be tested and at a preset height from the horizontal end face of the river section to be tested.

[0040] Specifically, two methods can be used to acquire the target video file. The first method involves a drone flying over the river section to be measured and using its camera to capture images of the river section from a frontal angle. A segment of the captured video, equal to the target duration, is then selected as the target video file. The second method involves setting up a camera at a predetermined height on one side of the river section, at a predetermined distance from the horizontal end face of the river section. The camera is then used to capture images of the river section from an oblique side angle. A segment of the captured video, equal to the target duration, is selected as the first video file. A perspective transformation is then performed on the first video file to obtain the target video file. The perspective transformation converts the video images in the first video file onto the target plane.

[0041] In the above-described implementation scheme of this application, the shooting position when acquiring the target video image can be one of two types: one is directly above the river section to be measured, and the other is obliquely to the side of the river section to be measured. The shooting method is more flexible, and the appropriate shooting method can be selected according to the actual geographical location of the river section to be measured.

[0042] In an optional embodiment of this application, when the target video file is obtained using the second method described above, obtaining the target video file obtained by performing perspective transformation on the first video file includes:

[0043] The perspective transformation relationship between a video image obtained by shooting the river section under test at the preset position and a video image obtained by shooting the river section under test from directly above the river section under test is obtained.

[0044] Based on the perspective transformation relationship, a perspective transformation is performed on each frame of the video image corresponding to the first video file to obtain the target video file.

[0045] Specifically, when the second method is used to obtain the target video file, since the oblique side shooting angle is used, the image in the video file obtained directly will be distorted compared with the content in the actual image. It is necessary to predetermine the perspective transformation relationship to convert the first video file into the target video file, and then perform perspective transformation on each frame of the video image corresponding to the first video file based on the perspective transformation relationship to obtain the target video file. This allows the oblique side shooting angle shooting method to also obtain a video file composed of the front view of the river section to be measured.

[0046] The above-described implementation scheme of this application obtains the perspective transformation relationship between a video image obtained by shooting the river section under test at a preset position and a video image obtained by shooting the river section under test directly above it. Then, the first video file is processed based on the perspective transformation relationship to obtain the target video file. This can achieve the purpose of shooting the river section under test from an oblique side to obtain the target video file.

[0047] In an optional embodiment of this application, obtaining the perspective transformation relationship between a video image obtained by photographing the river section to be measured at the preset location and a video image obtained by photographing the river section to be measured directly above it includes:

[0048] Obtain the quadrilateral corresponding to the first rectangle in the video image of the first video file, and mark the four vertices of the first rectangle on the horizontal end face of the river section to be measured.

[0049] The positions of each pixel in the quadrilateral are corrected to obtain a second rectangle with a shape similar to the first rectangle;

[0050] The perspective transformation relationship is determined based on the pixel position correspondence between the quadrilateral and the second rectangle.

[0051] Specifically, when obtaining the perspective transformation relationship between a video image captured from a preset location of the river section under test and a video image captured from directly above the river section under test, a rectangle (the first rectangle) can be pre-marked on the horizontal end face of the river section under test, i.e., on the ground. Specifically, only the four vertices of this rectangle need to be marked. The position and size of the rectangle need to be sufficient for clear imaging when shooting from the preset location; this application does not impose specific limitations. When the camera captures the river section under test from the preset location, the four vertices of the first rectangle in the resulting video image will form a new quadrilateral. The positions of each pixel in the quadrilateral are then corrected to obtain a second rectangle similar in shape to the first rectangle, with the same aspect ratio. Based on the correspondence between the pixel positions in the quadrilateral and the second rectangle, the perspective transformation relationship can be determined.

[0052] The above-described implementation scheme of this application marks a first rectangle on the horizontal end face of the river section to be measured, and corrects the pixel positions of the corresponding quadrilaterals in the video image of the first video file to obtain a second rectangle with a similar shape to the first rectangle. Thus, the perspective transformation relationship can be determined based on the pixel position correspondence between the quadrilaterals and the second rectangle. Since this perspective transformation relationship is obtained by analyzing and processing the rectangles marked on the ground and the captured images, the perspective transformation relationship corresponding to the change in shooting angle can be obtained based on this method when the shooting angle changes. In other words, the perspective transformation relationship is adjusted in real time with the change in shooting angle, thereby preventing the inaccuracy of the measurement results due to changes in the camera angle and further improving the accuracy of the measurement results.

[0053] In an optional embodiment of this application, the measurement method further includes:

[0054] Obtain the target distance between two marker points on the plane where the river section to be measured is located, wherein the direction of the line connecting the two marker points is consistent with the direction of the river width of the river section to be measured;

[0055] Obtain the first and second positions of the two marker points in the video image of the target video file, and obtain the number of target pixels between the first and second positions;

[0056] The pixel accuracy in the width direction of the river segment to be measured is determined based on the target distance and the number of target pixels.

[0057] To obtain the exposed width of the river section to be measured, it is also necessary to determine the ratio between the video image corresponding to the target video file and the actual river section, that is, the pixel accuracy. Since this embodiment obtains the exposed width of the river, it is only necessary to obtain the pixel accuracy in the direction of the width of the river section to be measured.

[0058] Specifically, two points can be marked on the plane where the river section to be measured is located. The line connecting these two points is in the same direction as the river width. Then, the target distance between these two points, as well as the first and second positions of these two points in the video image corresponding to the target video file, are obtained. The number of target pixels between the first and second positions is also obtained. The ratio of the target distance to the number of target pixels is the pixel precision, which represents the actual length corresponding to one pixel in the video image corresponding to the target video image.

[0059] The above-described implementation scheme of this application can determine the pixel accuracy by obtaining the target distance between two marker points and the number of target pixels between the first position and the second position. Based on this pixel accuracy, the open river width corresponding to the river section to be measured can be determined in subsequent processing.

[0060] In an optional embodiment of this application, step 104, based on the target video image, the target mask image, and the target ice floe trajectory point set, determines the exposed river width corresponding to the river section to be measured, including:

[0061] Based on the target video image, the target mask image, and the target ice track point set, determine the number of first pixel points corresponding to the exposed river width of the river section to be measured in the river width direction;

[0062] The width of the open river is determined based on the number of the first pixels and the pixel precision.

[0063] Specifically, after determining the number of first pixels corresponding to the exposed river width in the river width direction of the river segment to be measured based on the target video image, target mask image, and target ice track point set, it is also necessary to determine the specific length of the exposed river width based on the number of first pixels and pixel precision.

[0064] Specifically, determining the number of first pixels corresponding to the exposed river width of the river segment under test in the river width direction based on the target video image, the target mask image, and the target ice floe trajectory point set includes:

[0065] Obtain the number of second pixels corresponding to the target video image in the direction of the river width;

[0066] The pixels of the target video image are detected row by row from top to bottom along the width of the river. The detection stops when the pixels in the Nth row meet the preset conditions, where N is an integer greater than or equal to 1. The preset conditions are that all pixels in the row do not belong to the mask part corresponding to the target mask image, or there are pixels in the row that overlap with the target ice track point set.

[0067] The pixels of the target video image are detected row by row from bottom to top along the width of the river. The detection stops when the pixels in the Mth row meet the preset condition, where M is an integer greater than or equal to 1.

[0068] Based on the number of the second pixel points, the Nth row, and the Mth row, determine the number of the first pixel points corresponding to the exposed river width of the river segment to be measured in the river width direction.

[0069] In this embodiment, when determining the number of pixels corresponding to the river section to be tested in the river width direction, which is the first number of pixels, the second number of pixels corresponding to the target video image in the river width direction is first determined. Then, the target video image is detected sequentially from top to bottom and from bottom to top in the river width direction. The detection stops when a preset condition is met, that is, when the row is not shore ice. The corresponding Nth row and Mth row can be obtained. That is, the first row from top to bottom to the Nth row are shore ice or ground, and the last row to the last Mth row are shore ice or ground. The part between the Nth row and the Mth row is not shore ice. This part is the water flow with ice floating on it, which is the exposed part of the river section to be tested.

[0070] Specifically, based on the number of second pixels, the Nth row, and the Mth row, the number of first pixels corresponding to the exposed river width in the river width direction of the river segment to be tested can be determined, wherein the number of first pixels is equal to the number of second pixels minus N and M.

[0071] It should be noted that since shore ice is a continuous and stationary ice surface, if all pixels in a row do not belong to the mask portion corresponding to the target mask image, or if there are pixels in that row that overlap with the target ice trajectory point set, then it can be determined that the row meets the preset conditions. In other words, if the row meets the preset conditions, then the row does not belong to shore ice.

[0072] The above-described implementation scheme of this application, by combining the target mask map and the target ice icicle trajectory point set, detects the target video image line by line, which can determine the number of pixels corresponding to the exposed part of the river section to be measured in the river width direction. This realizes the goal of obtaining the target exposed river width by analyzing and processing the video file acquired by the camera, reducing the degree of human involvement in the measurement process and improving the accuracy of the exposed river width corresponding to the river section to be measured.

[0073] In an optional embodiment of this application, icicle recognition is performed on each frame of video image in the target video file to obtain a set of target icicle trajectory points, including:

[0074] Step A1: For each frame of video image in the target video file, obtain the ice crystal feature points corresponding to the current video image.

[0075] Ice formations are a natural phenomenon that occurs in rivers under low-temperature conditions. When the water temperature drops below zero degrees Celsius, slowly flowing river water begins to form ice flowers and icicles, which float and aggregate on the surface, varying in size and shape. Ice feature points are used to characterize ice features in video images. For example, the pixel representing one vertex of an ice icicle in a video image is called the ice feature point of that vertex. The location of the ice icicle in the video image can be determined by the position of these feature points. For each frame of a target video file, multiple ice feature points corresponding to the current video image can be obtained through feature recognition.

[0076] The above-mentioned acquisition of ice crystal feature points corresponding to each frame of video image in the target video file includes:

[0077] For each frame of video image, the second semantic segmentation model is used to perform semantic segmentation on the current video image to obtain the corresponding ice mask;

[0078] Identify a first feature point in the current video image that meets preset conditions;

[0079] The first feature point located within the ice mask in the current video image is determined as the ice feature point corresponding to the current video image.

[0080] Semantic segmentation models are used to classify each pixel of an image, dividing the image into several visually meaningful or regions of interest. In this embodiment, the second semantic segmentation model, through pre-training, can segment icicles in the image as regions of interest. Therefore, the second semantic segmentation model can be used to perform semantic recognition on each frame of the video image to obtain an icicle mask for each frame. The icicle mask represents the region where the icicles are located in the video image. Simultaneously, feature point recognition is performed on each frame of the video image, that is, identifying feature points in the current video image that meet preset conditions to obtain first feature points. Feature points that meet the preset conditions are points where the image grayscale value changes drastically or points with large curvature on the image edge. For example, SIFT (Scale-invariant feature transform) calculation can be performed on each frame of the video image. Through SIFT calculation, the feature points in the video image and their related dimensions, orientations, etc., can be determined. The first feature points identified and located within the icicle mask are determined as icicle feature points. Therefore, icicle feature points for each frame of the video image can be obtained in the above manner, and there can be multiple icicle feature points.

[0081] Step A2: For each frame of video image, match the ice feature points of the current video image with the set of predicted ice trajectory points, and determine the set of ice trajectory points corresponding to the current video image based on the matching result. Each set of ice trajectory points contains ice trajectory points between the time corresponding to the first frame of the target video image and the time corresponding to the current video image. The predicted ice trajectory points are predicted based on the ice feature points corresponding to the previous frame of the current video image.

[0082] After obtaining multiple video images, the video images can be sorted according to the time sequence of shooting. Based on the ice feature points of the video images with earlier times, the ice trajectory points of the video images at the next moment can be predicted.

[0083] For example, after obtaining the icicle feature points of the first frame video image, these icicle feature points can be identified as icicle trajectory points and added to the corresponding icicle trajectory point sets. Specifically, since there is no preceding video image for the first frame video image, meaning that the icicle feature points at this time do not have corresponding predicted trajectory points, the icicle feature points of the first frame video image can be identified as icicle trajectory points, and an icicle trajectory point set can be configured for each icicle trajectory point, adding the icicle trajectory points to the corresponding icicle trajectory point sets.

[0084] For each set of ice icicle trajectory points, Kalman filtering can be applied to these points. Since Kalman filtering is an algorithm that predicts the position of the next moment based on the combined measurements of the previous moment's position, the predicted trajectory points of the ice icicles corresponding to each set of ice icicle trajectory points in the second frame of the video image can be predicted. When the current video image is the second frame, the predicted trajectory points obtained from the ice icicle trajectory point sets are the predicted ice icicle trajectory points.

[0085] By performing semantic recognition and feature point recognition on the video image as described above, the ice crystal feature points of the video image can be obtained. When the current video image is the second frame, the current ice crystal feature points of the current video image can be obtained by performing semantic recognition and feature point recognition on the second frame.

[0086] Because icicles are fluid in rivers, the position of the same icicle in the first video frame will differ from its position in the second video frame. Furthermore, since multiple icicles exist in the video image, to determine the position of an icicle in the first video frame in the second video frame, this embodiment matches the current icicle feature points obtained from semantic and feature point recognition of the second video frame with the predicted icicle trajectory points predicted based on the icicle feature points of the first video frame. The matching results determine the current icicle trajectory point set. Each current icicle trajectory point set contains the icicle trajectory points of the corresponding icicle at different times.

[0087] In some optional embodiments of this application, each icicle feature point has a corresponding feature score, which characterizes the prominence of the corresponding icicle feature point in the image. Generally, the higher the feature score, the more prominent the corresponding icicle feature point, and the greater the probability that the icicle feature point is a point on the icicle. Conversely, the lower the feature score, the less prominent the corresponding icicle feature point, and the smaller the probability that the icicle feature point is a point on the icicle, i.e., the greater the probability that the icicle feature point is an interference point. In a specific implementation, when calculating the feature points of the video image using SIFT, the feature score of each feature point can be obtained accordingly.

[0088] The above process involves matching the ice feature points of the current video image with the predicted ice trajectory point set, and determining the ice trajectory point set corresponding to the current video image based on the matching results, including:

[0089] For the current video image, the ice icicle feature points are classified based on the feature scores corresponding to the ice icicle feature points to obtain a first type of ice icicle feature points and a second type of ice icicle feature points. The feature scores corresponding to the first type of ice icicle feature points are greater than a first preset feature score, and the feature scores corresponding to the second type of ice icicle feature points are less than or equal to the first preset feature score.

[0090] The first type of ice feature points are matched with the set of predicted ice trajectory points to determine the first type of ice feature points that are successfully matched and the first type of ice feature points that are not successfully matched, as well as a first subset including the predicted ice trajectory points that are not successfully matched with the first type of ice feature points.

[0091] Determine whether the feature score of the first type of ice crystal feature point that failed to match is greater than the second preset feature score, wherein the first preset feature score is greater than the second preset feature score;

[0092] If so, the first type of ice feature points that failed to match are added to the ice trajectory point set corresponding to the current video image;

[0093] The first subset is matched with the second type of ice feature points to determine the successfully matched second type of ice feature points and a second subset including the predicted ice trajectory points that did not match the second type of ice feature points.

[0094] For each predicted ice track point in the second subset, determine whether the number of consecutive unmatched ice track points in the set corresponding to the second ice track point is greater than the number threshold.

[0095] If not, the unmatched second predicted ice track point is added to the corresponding ice track point set;

[0096] The first type of icicle feature points and the second type of icicle feature points that are successfully matched are added to the corresponding current icicle trajectory point set to obtain the icicle trajectory point set corresponding to the current video image.

[0097] In this embodiment, the ice feature points corresponding to the current video image are classified. Ice feature points with feature scores greater than a first preset feature score are determined as first-class ice feature points, and ice feature points with feature scores less than or equal to the first preset feature score are determined as second-class ice feature points.

[0098] Specifically, the first type of ice feature points with higher feature scores are first matched with the preset set of ice trajectory points to determine the first type of ice feature points that are successfully matched, as well as the first type of ice feature points and predicted ice trajectory points that are not successfully matched. In order to facilitate the distinction from subsequent matching results, the set corresponding to the predicted ice trajectory points that are not successfully matched with the first type of ice feature points is denoted as the first subset.

[0099] A successful match means that the position of the ice icicle corresponding to the ice icicle trajectory point set has been found in the current video image, that is, the coordinate position of the first type of ice icicle feature point that was successfully matched. Therefore, the first type of ice icicle feature point that was successfully matched is added to the corresponding ice icicle trajectory point set as the ice icicle trajectory point at the current moment to update the ice icicle trajectory point set and obtain the current ice icicle trajectory point set.

[0100] For unmatched first-type icicle feature points, their feature scores can be used to determine whether they represent newly generated icicles. If they do, the unmatched first-type icicle feature point is added to the corresponding icicle trajectory point set. In other words, a corresponding icicle trajectory point set is configured for each unmatched first-type icicle feature point, and the feature point is added to the set. If it does not represent a newly generated icicle, the unmatched first-type icicle feature point can be discarded.

[0101] In one example, determining whether a newly generated icicle is represented by its feature score based on the feature score of a first-type icicle feature point that failed to match can be achieved by comparing the feature score with a second preset feature score. If the feature score is greater than the second preset feature score, it is considered to represent a newly generated icicle; if it is less than or equal to the second preset feature score, it is considered not to represent a newly generated icicle. In another example, determining whether a newly generated icicle is represented by its feature score based on the feature score of a first-type icicle feature point that failed to match can be achieved by obtaining the ranking of the feature scores of the first-type icicle feature point among multiple current icicle feature points in the current video image. This ranking is based on the feature scores from largest to smallest. If the ranking is within a preset ranking, the first-type icicle feature point is considered to represent a newly generated icicle; if the ranking exceeds the preset ranking, it is considered not to represent a newly generated icicle.

[0102] For a first subset including predicted ice icicle trajectory points that did not match the first type of ice icicle feature points, further matching with the second type of ice icicle feature points is performed to determine the successfully matched second type of ice icicle feature points, and to determine the unmatched second type of ice icicle feature points and a second subset including the unmatched predicted ice icicle trajectory points.

[0103] A successful match means that the position of the ice icicle corresponding to the ice icicle trajectory point set in the current video image has been found, that is, the coordinate position of the successfully matched second type of ice icicle feature point. Therefore, the successfully matched second type of ice icicle feature point is added to the corresponding ice icicle trajectory point set as the ice icicle trajectory point at the current moment to update the ice icicle trajectory point set and obtain the current ice icicle trajectory point set.

[0104] Since the probability of the second type of ice feature point being represented as a point on the ice is relatively small, the second type of ice feature points that fail to match can be discarded directly.

[0105] Considering that some edges / vertices may be occluded during the flow of icicles, resulting in the icicle feature points not being identified in the corresponding video image, for the predicted icicle trajectory points in the second subset, the number of unmatched attempts in the corresponding icicle trajectory point set can be recorded. That is, the number of unmatched attempts in the corresponding icicle trajectory point set can be incremented by 1, and it can be determined whether the number of unmatched attempts in the corresponding icicle trajectory point set is equal to the threshold. If yes, it is considered that the icicle corresponding to the icicle trajectory point set has disappeared; if no, it is considered that the icicle corresponding to the icicle trajectory point set has been occluded in the short term. In this case, the icicle trajectory point is added to the corresponding icicle trajectory point set to update the icicle trajectory point set and obtain the current icicle trajectory point set.

[0106] Since the predicted ice icicle trajectory points in the second subset do not have matching ice icicle feature points, it is difficult to intuitively see their specific locations in the video image. Therefore, it can be considered that the ice icicle trajectory point set with the added predicted ice icicle trajectory points in the second subset is invisible in the current video image.

[0107] Step A3: Determine the set of current ice icicle trajectory points corresponding to the last frame of the target video image as the target ice icicle trajectory point set.

[0108] Specifically, the set of target ice icicle trajectory points is the combination of the ice icicle trajectory points corresponding to the last frame of the target video image.

[0109] The measurement method provided by the embodiments of this application has been described above. The measurement device provided by the embodiments of this application will be described below with reference to the accompanying drawings.

[0110] like Figure 2 As shown, this embodiment of the invention also provides a measuring device, the device comprising:

[0111] The first acquisition module 201 is used to acquire a target video file corresponding to the river section to be tested within a target duration. The target video file includes a video image of the river section to be tested on a target plane. The target plane is parallel to the horizontal end face of the river section to be tested.

[0112] The second acquisition module 202 is used to process the target video image based on the first semantic segmentation model to obtain the target mask image corresponding to the target video image. The target video image is any frame video image in the target video file, and the mask part corresponding to the target mask image includes shore ice and ice floes.

[0113] The third acquisition module 203 is used to perform ice crystal recognition on each frame of video image in the target video file and acquire a set of target ice crystal trajectory points;

[0114] The first determining module 204 is used to determine the open river width corresponding to the river section to be measured based on the target video image, the target mask image, and the target ice track point set.

[0115] Optionally, the first acquisition module 201 includes one of the following sub-modules:

[0116] The first acquisition submodule is used to acquire the target video file corresponding to the target duration, which is captured directly above the river section to be measured;

[0117] The second acquisition submodule is used to acquire the target video file obtained by performing perspective transformation on the first video file. The first video file is a video file corresponding to the target duration obtained by shooting the river section to be tested at a preset position. The preset position is a position located on one side of the river section to be tested and at a preset height from the horizontal end face of the river section to be tested.

[0118] Optionally, the second acquisition submodule includes:

[0119] The first acquisition unit is used to acquire the perspective transformation relationship between a video image obtained by shooting the river section to be tested at the preset position and a video image obtained by shooting the river section to be tested from directly above the river section to be tested.

[0120] The processing unit is configured to perform perspective transformation on each frame of the video image corresponding to the first video file based on the perspective transformation relationship, so as to obtain the target video file.

[0121] Optionally, the acquisition unit includes:

[0122] The acquisition subunit is used to acquire the quadrilateral corresponding to the first rectangle in the video image of the first video file, and the four vertices of the first rectangle are marked on the horizontal end face of the river section to be measured.

[0123] A processing subunit is used to correct the position of each pixel in the quadrilateral to obtain a second rectangle with a shape similar to the first rectangle;

[0124] A subunit is defined to determine the perspective transformation relationship based on the pixel position correspondence between the quadrilateral and the second rectangle.

[0125] Optionally, the measuring device further includes:

[0126] The fourth acquisition module is used to acquire the target distance between two marker points on the plane where the river section to be measured is located, wherein the direction of the line connecting the two marker points is consistent with the direction of the river width of the river section to be measured;

[0127] The fifth acquisition module is used to acquire the first position and the second position corresponding to the two marker points in the video image of the target video file, and to acquire the number of target pixels between the first position and the second position;

[0128] The second determining module is used to determine the pixel accuracy in the river width direction of the river section to be measured based on the target distance and the number of target pixels.

[0129] Optionally, the first determining module 204 includes:

[0130] The first determining submodule is used to determine the number of first pixels corresponding to the exposed river width of the river section to be measured in the river width direction based on the target video image, the target mask image and the target ice track point set;

[0131] The second determining submodule is used to determine the width of the exposed river based on the number of the first pixels and the pixel precision.

[0132] Optionally, the first determining submodule includes:

[0133] The second acquisition unit is used to acquire the number of second pixels corresponding to the target video image in the river width direction;

[0134] The first detection unit is used to detect the pixels of the target video image row by row from top to bottom along the width of the river. The detection stops when the pixels in the Nth row meet the preset conditions, where N is an integer greater than or equal to 1. The preset conditions are that all pixels in the row do not belong to the mask part corresponding to the target mask image, or there are pixels in the row that overlap with the target ice track point set.

[0135] The second detection unit is used to detect the pixels of the target video image row by row from bottom to top along the width of the river, and to stop detection when the pixels in the Mth row meet the preset condition, where M is an integer greater than or equal to 1.

[0136] The determining unit is configured to determine, based on the number of the second pixel points, the Nth row, and the Mth row, the number of the first pixel points corresponding to the exposed river width of the river segment to be measured in the river width direction.

[0137] Optionally, the third acquisition module 203 includes:

[0138] The third acquisition submodule is used to acquire multiple ice crystal feature points corresponding to each frame of video image in the target video file.

[0139] The matching submodule is used to match the ice feature points of the current video image with the predicted ice trajectory point set for each frame of the video image, and determine the ice trajectory point set corresponding to the current video image based on the matching result. Each ice trajectory point set contains ice trajectory points between the time corresponding to the first frame of the target video image and the time corresponding to the current video image. The predicted ice trajectory points are predicted based on the ice feature points corresponding to the previous frame of the current video image.

[0140] The third determining submodule is used to determine the set of ice icicle trajectory points corresponding to the last frame of the target video image as the target ice icicle trajectory point set.

[0141] Optionally, the third acquisition submodule includes:

[0142] The segmentation unit is used to perform semantic segmentation on the current video image for each frame of video image using the second semantic segmentation model to obtain the ice mask corresponding to the current video image.

[0143] The recognition unit is used to identify a first feature point in the current video image that meets preset conditions;

[0144] The determining unit is used to determine the first feature point located within the ice mask in the current video image as the ice feature point corresponding to the current video image.

[0145] Optionally, each of the icicle feature points has a corresponding feature score;

[0146] The matching submodule is further used for:

[0147] For the current video image, the ice icicle feature points are classified based on the feature scores corresponding to the ice icicle feature points to obtain a first type of ice icicle feature points and a second type of ice icicle feature points. The feature scores corresponding to the first type of ice icicle feature points are greater than a first preset feature score, and the feature scores corresponding to the second type of ice icicle feature points are less than or equal to the first preset feature score.

[0148] The first type of ice feature points are matched with the set of predicted ice trajectory points to determine the first type of ice feature points that are successfully matched and the first type of ice feature points that are not successfully matched, as well as a first subset including the predicted ice trajectory points that are not successfully matched with the first type of ice feature points.

[0149] Determine whether the feature score of the first type of ice crystal feature point that failed to match is greater than the second preset feature score, wherein the first is greater than the second;

[0150] If so, the first type of ice feature points that failed to match are added to the ice trajectory point set corresponding to the current video image;

[0151] The first subset is matched with the second type of ice feature points to determine the successfully matched second type of ice feature points and a second subset including the predicted ice trajectory points that did not match the second type of ice feature points.

[0152] For each predicted ice track point in the second subset, determine whether the number of consecutive unmatched ice track points in the set corresponding to the second ice track point is greater than the number threshold.

[0153] If not, the unmatched second predicted ice track point is added to the corresponding ice track point set;

[0154] The first type of icicle feature points and the second type of icicle feature points that are successfully matched are added to the corresponding current icicle trajectory point set to obtain the icicle trajectory point set corresponding to the current video image.

[0155] The measuring device provided in this application processes the target video file corresponding to the river section to be measured to obtain the target video image, the target mask image including the bank ice and ice floes, and the target ice floe trajectory point set, thereby determining the exposed river width of the river section to be measured. This achieves the purpose of measuring the exposed river width by processing the video file, reduces the degree of human involvement in the measurement process, and thus improves the accuracy of the exposed river width corresponding to the river section to be measured.

[0156] This application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described measurement method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here.

[0157] For example, Figure 3 A schematic diagram of the physical structure of an electronic device is shown.

[0158] like Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 can call logical instructions in the memory 330. The processor 310 is used to perform the following steps: acquiring a target video file corresponding to the river segment to be tested within a target duration, the target video file including video images of the river segment to be tested on a target plane, the target plane being parallel to the horizontal end face of the river segment to be tested; processing the target video images based on a first semantic segmentation model to acquire a target mask image corresponding to the target video image, the target video image being any frame video image in the target video file, the mask portion corresponding to the target mask image including bank ice and ice floes; performing ice floe recognition on each frame video image in the target video file to acquire a set of target ice floe trajectory points; determining the open river width corresponding to the river segment to be tested based on the target video image, the target mask image, and the set of target ice floe trajectory points.

[0159] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0160] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described measurement method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0161] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0163] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

[0164] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0165] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0166] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0167] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0169] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0170] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method of measurement, characterized by, The method comprises the following steps: acquiring a target video file corresponding to a target time length of a to-be-tested river section, the target video file comprising video images corresponding to a target plane of the to-be-tested river section, the target plane being parallel to a horizontal end surface of the to-be-tested river section; processing the target video images based on a first semantic segmentation model to acquire target masks corresponding to the target video images, the target video images being any frame video images in the target video file, and the mask parts corresponding to the target masks comprising shore ice and ice floes; performing ice floe identification on each frame video image in the target video file to acquire a target ice floe track point set; determining an open river width corresponding to the to-be-tested river section according to the target video images, the target masks and the target ice floe track point set.

2. The measurement method according to claim 1, characterized in that, The acquiring of the target video file corresponding to the target time length of the to-be-tested river section comprises one of the following solutions: acquiring the target video file corresponding to the target time length, which is shot directly above the to-be-tested river section; acquiring the target video file obtained by performing perspective transformation on a first video file, the first video file being a video file corresponding to the target time length, which is obtained by shooting the to-be-tested river section at a preset position, the preset position being a position located on one side of the to-be-tested river section and being located at a preset height from the horizontal end surface of the to-be-tested river section.

3. The measurement method according to claim 2, characterized in that, The acquiring of the target video file obtained by performing perspective transformation on the first video file comprises: acquiring a perspective transformation relationship between a video image obtained by shooting the to-be-tested river section at the preset position and a video image obtained by shooting the to-be-tested river section directly above the to-be-tested river section; performing perspective transformation on each frame video image corresponding to the first video file based on the perspective transformation relationship to obtain the target video file.

4. The measurement method according to claim 2, characterized by, The method further comprises the following steps: acquiring a target distance between two marker points marked on a plane where the to-be-tested river section is located, the direction of the line segment between the two marker points being consistent with the river width direction of the to-be-tested river section; acquiring a first position and a second position of the two marker points in the video images of the target video file and acquiring a target number of pixel points between the first position and the second position; determining a pixel accuracy in the river width direction of the to-be-tested river section according to the target distance and the target number of pixel points.

5. The measurement method according to claim 4, characterized in that, The determining of the open river width corresponding to the to-be-tested river section according to the target video images, the target masks and the target ice floe track point set comprises: determining a first number of pixel points of the open river width corresponding to the to-be-tested river section in the river width direction according to the target video images, the target masks and the target ice floe track point set; determining the open river width according to the first number of pixel points and the pixel accuracy.

6. The measurement method according to claim 5, characterized in that, The determining of the first number of pixel points of the open river width corresponding to the to-be-tested river section in the river width direction according to the target video images, the target masks and the target ice floe track point set comprises: acquiring a second number of pixel points corresponding to the target video images in the river width direction; Detecting the pixel points of the target video image from top to bottom along the river width direction, and stopping the detection when the pixel points of the Nth row meet the preset condition, N being an integer greater than or equal to 1, wherein all the pixel points of the row meeting the preset condition do not belong to the mask part corresponding to the target mask image, or there are pixel points in the row that coincide with the target ice and snow trajectory point set; Detecting the pixel points of the target video image from bottom to top along the river width direction, and stopping the detection when the pixel points of the Mth row meet the preset condition, M being an integer greater than or equal to 1; According to the second pixel point number, the Nth row and the Mth row, determining the first pixel point number corresponding to the open river width of the river section to be measured in the river width direction.

7. The measurement method according to claim 1, characterized by, Ice and snow identification is performed on each video image in the target video file to obtain a target ice and snow trajectory point set, including: For each video image in the target video file, ice and snow feature points corresponding to the current video image are obtained; For each video image, the ice and snow feature points of the current video image are matched with a predicted ice and snow trajectory point set, and the ice and snow trajectory point set corresponding to the current video image is determined according to the matching result, each ice and snow trajectory point set containing ice and snow trajectory points between the time corresponding to the first video image in the target video image and the time corresponding to the current video image, the predicted ice and snow trajectory point being predicted based on the ice and snow feature points corresponding to the previous video image of the current video image; The ice and snow trajectory point set corresponding to the last video image in the target video image is determined as the target ice and snow trajectory point set.

8. A measuring device, characterized by It includes: The first acquisition module is used for acquiring a target video file corresponding to a river section to be measured within a target time length, the target video file including video images corresponding to the river section to be measured on a target plane, the target plane being parallel to the horizontal end surface of the river section to be measured; The second acquisition module is used for processing the target video image based on a first semantic segmentation model to obtain a target mask image corresponding to the target video image, the target video image being any video image in the target video file, and the mask part corresponding to the target mask image including bank ice and ice and snow; The third acquisition module is used for performing ice and snow identification on each video image in the target video file to obtain a target ice and snow trajectory point set; The first determination module is used for determining an open river width corresponding to the river section to be measured according to the target video image, the target mask image and the target ice and snow trajectory point set.

9. An electronic device, comprising: The computer program is stored on the computer readable storage medium and is executed by the processor to implement the steps of the measurement method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium and is executed by the processor to implement the steps of the measurement method according to any one of claims 1 to 7.

Images