River surface flow velocity identification data enhancement method and device

Abstract

The invention provides a river surface flow velocity identification data enhancement method and device, and belongs to the technical field of hydrological flow measurement, and the river surface flow velocity identification data enhancement method comprises the steps: obtaining a river data set; the flow velocity label is mapped to a gray value interval to obtain a gray value label graph, and the gray value of any point in the gray value label graph corresponds to the flow velocity; and randomly selecting a region of interest in each river original image and the gray scale label image, cutting the region of interest to obtain a cut image, and taking the cut image as training data of a deep learning model. According to the river surface flow velocity identification data enhancement method, the flow velocity label is mapped to the gray value, and the gray label image is made, so that compared with a traditional flow velocity label image or an optical flow image, the utilization efficiency of deep learning training data can be improved, and meanwhile, the utilization rate of the deep learning training data is improved. And different image receptive fields are provided by adopting a random cutting mode to improve the robustness of model training.

Description

Technical Field

[0001] The present invention relates to the technical field of hydrological flow measurement, and particularly to a method and device for enhancing river surface velocity identification data. Background Art

[0002] With the development of artificial intelligence and video surveillance technologies, non-contact measurement of river velocity has become possible. By capturing images of the river surface and using machine vision and image processing technologies, the modes and tracers (such as floating objects, water surface ripples, etc.) in the fluid can be identified, and their movement speeds can be calculated, thereby inferring the river velocity. This technology has the advantages of real-time, accuracy, and environmental friendliness, and can achieve full-field and dynamic velocity measurement without disturbing the river ecological environment. Based on the identification results, the dispatching department can combine GIS (Geographic Information System) map information and river management requirements to achieve real-time early warning and decision support.

[0003] When using a deep learning-based algorithm for river surface velocity identification tasks, collecting a high-quality river dataset with velocity labels is crucial. The existing methods for collecting river datasets are all based on images with a fixed resolution in monitoring devices, and then binding them with the measured actual surface velocity values. Although this method of making label data can correctly train the model in the deep learning manner, due to the single scene and small number of labels of the original data, it is difficult to train a high-precision velocity identification model. Summary of the Invention

[0004] In view of this, the present invention provides a method and device for enhancing river surface velocity identification data, which are used to solve the problems that the dataset used by the current deep learning model has a single scene, a small number of labels, and it is difficult to train a high-precision velocity identification model.

[0005] To solve the above technical problems, in the first aspect, the present invention provides a method for enhancing river surface velocity identification data, the method comprising:

[0006] Obtaining a river dataset, the river dataset including a river original image and a velocity label corresponding to the river original image;

[0007] Mapping the velocity label to a gray value interval to obtain a gray label map, wherein the gray value of any point in the gray label map corresponds to the velocity. The step of mapping the velocity label to a gray value interval to obtain a gray label map includes: if there is a point with a gray value of 0 after mapping the velocity label to the gray value interval, using the K-nearest neighbor algorithm to find the nearest neighboring point with a non-zero gray value, and filling the point with a gray value of 0 with the neighboring point.

[0008] Randomly select regions of interest in each of the original river maps and the grayscale label maps and perform cropping to obtain cropped images, which are used as training data for the deep learning model. Among them, the positions and sizes of the regions of interest selected in the original river map and the corresponding grayscale label map are the same.

[0009] Optionally, the step of randomly selecting regions of interest in each of the original river maps and the grayscale label maps and performing cropping to obtain cropped images includes:

[0010] Randomly determine the size of the cropping frame corresponding to the region of interest, and the cropping frame is a square;

[0011] Obtain the width and height of the river channel region in the grayscale label map, as well as the position coordinates of the upper left corner and the lower right corner of the river channel region in the grayscale label map;

[0012] According to the size relationship between the width and height of the river channel region, determine the image type of the river channel region image. The river channel region image is obtained by intercepting the river channel region from the grayscale label map, and the image type includes horizontal images and vertical images;

[0013] According to the image type, the width and height of the river channel region, the position coordinates of the upper left corner and the lower right corner of the river channel region in the grayscale label map, and the size of the cropping frame, determine the coordinate range of the upper left corner of the region of interest;

[0014] According to the coordinate range of the upper left corner of the region of interest, crop the original river map and the grayscale label map to obtain cropped images.

[0015] Optionally, the step of determining the coordinate range of the upper left corner of the region of interest according to the image type, the width and height of the river channel region, the position coordinates of the upper left corner and the lower right corner of the river channel region in the grayscale label map, and the size of the cropping frame includes:

[0016] If the image type is a horizontal image, the horizontal coordinate range of the upper left corner of the region of interest is [0, , and the vertical coordinate range is , ;

[0017] If the image type is a vertical image, the horizontal coordinate range of the upper left corner of the region of interest is , , and the vertical coordinate range is [0, ;

[0018] Wherein, w is the width of the river channel area, h is the height of the river channel area, l and t are the abscissa and ordinate of the upper left corner of the river channel area in the grayscale label map respectively, and r and b are the abscissa and ordinate of the lower right corner of the river channel area in the grayscale label map respectively. is the width of the cropping frame, is the height of the cropping frame, is the preset height range value when the image type is a landscape image, is the preset width range value when the image type is a portrait image.

[0019] Optionally, before the step of cropping the original river image and the grayscale label map according to the coordinate range of the upper left corner of the region of interest to obtain a cropped image, the method further includes:

[0020] If the size of the cropping frame is larger than the size of the river channel area, then boundary filling is performed on the part of the region of interest in the original river image and the grayscale label map that exceeds the river channel area using pixels with a grayscale value of 0.

[0021] Optionally, the determining the image type of the river channel area image according to the size relationship between the width and height of the river channel area includes:

[0022] If the width of the river channel area is greater than the height, then it is determined that the image type is a landscape image;

[0023] If the width of the river channel area is less than or equal to the height, then it is determined that the image type is a portrait image.

[0024] In a second aspect, the present invention further provides a river surface velocity recognition data enhancement device, and the device includes:

[0025] An acquisition module, configured to acquire a river data set, where the river data set includes an original river image and a velocity label corresponding to the original river image;

[0026] A mapping module, configured to map the velocity label to a grayscale value interval to obtain a grayscale label map, where the grayscale value of any point in the grayscale label map corresponds to the velocity. The mapping module includes a filling unit, and the filling unit is configured to, if there is a point with a grayscale value of 0 after mapping the velocity label to the grayscale value interval, use the K-nearest neighbor algorithm to find the nearest neighboring point with a non-zero grayscale value, and use the neighboring point to fill the point with a grayscale value of 0.

[0027] A cropping module for randomly selecting regions of interest in each of the original river images and the grayscale label images, cropping them to obtain cropped images, and using the cropped images as training data for a deep learning model. Among them, the positions and sizes of the regions of interest selected from the original river image and the corresponding grayscale label image are the same;

[0029] Optionally, the cropping module includes:

[0030] A first determination unit for randomly determining the size of the cropping frame corresponding to the region of interest, where the cropping frame is a square;

[0031] An acquisition unit for acquiring the width and height of the river channel area in the grayscale label image, as well as the position coordinates of the upper left corner and the lower right corner of the river channel area in the grayscale label image;

[0032] A judgment unit for judging the image type of the river channel area image according to the size relationship between the width and height of the river channel area. The river channel area image is obtained by intercepting the river channel area from the grayscale label image, and the image type includes horizontal images and vertical images;

[0033] A second determination unit for determining the coordinate range of the upper left corner of the region of interest according to the image type, the width and height of the river channel area, the position coordinates of the upper left corner and the lower right corner of the river channel area in the grayscale label image, and the size of the cropping frame;

[0034] A cropping unit for cropping the original river image and the grayscale label image according to the coordinate range of the upper left corner of the region of interest to obtain cropped images.

[0035] Optionally, the second determination unit includes:

[0036] A first determination subunit for if the image type is a horizontal image, then the abscissa range of the upper left corner of the region of interest is [0, , and the ordinate range is , ;

[0037] A second determination subunit for if the image type is a vertical image, then the abscissa range of the upper left corner of the region of interest is , , and the ordinate range is [0, ;

[0038] Where, w is the width of the river channel area, h is the height of the river channel area, l and t are respectively the abscissa and ordinate of the upper left corner of the river channel area in the grayscale label map, and r and b are respectively the abscissa and ordinate of the lower right corner of the river channel area in the grayscale label map. is the width of the cropping frame, is the height of the cropping frame, is the preset height range value when the image type is a landscape image, is the preset width range value when the image type is a portrait image.

[0039] Optionally, the device further includes:

[0040] A filling module, configured to, if the size of the cropping frame is greater than the size of the river channel area, perform boundary filling on the part of the region of interest in the original river image and the grayscale label map that exceeds the river channel area with pixels having a grayscale value of 0.

[0041] Optionally, the judging unit includes:

[0042] A first judging subunit, configured to determine that the image type is a landscape image if the width of the river channel area is greater than the height;

[0043] A second judging subunit, configured to determine that the image type is a portrait image if the width of the river channel area is less than or equal to the height.

[0044] The beneficial effects of the above technical solutions of the present invention are as follows:

[0045] In the embodiments of the present invention, by mapping the flow velocity label to a grayscale value to produce a grayscale label image, the utilization efficiency of deep learning training data can be improved compared with traditional flow velocity label images or optical flow images; at the same time, different image receptive fields are provided by means of random cropping, which can improve the robustness of model training. Description of the Drawings

[0046] Figure 1 is a schematic flow chart of a method for enhancing river surface flow velocity recognition data in the first embodiment of the present invention;

[0047] Figure 2 is a schematic flow chart of randomly cropping an image in the first embodiment of the present invention

[0048] Figure 3 is a schematic diagram of the output result of the deep learning model in the first embodiment of the present invention;

[0049] Figure 4 is a schematic structural diagram of a device for enhancing river surface flow velocity recognition data in the second embodiment of the present invention. Detailed Embodiments

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the described embodiments of the present invention fall within the scope of protection of the present invention.

[0051] With the development of artificial intelligence and video surveillance technologies, it has become possible to perform non-contact measurement of river flow velocity. By capturing images of the river surface and using machine vision and image processing technologies, it is possible to identify the modalities and tracers (such as floating objects, water surface ripples, etc.) in the fluid and calculate their movement speeds, thereby inferring the river flow velocity. This technology has the characteristics of real-time, accuracy, and environmental friendliness, and can achieve full-field and dynamic flow velocity measurement without disturbing the river ecological environment. Based on the recognition results, the dispatching department can combine GIS (Geographic Information System) map information and river management requirements to achieve real-time early warning and decision support.

[0052] When using a deep learning-based algorithm to perform the river surface flow velocity recognition task, it is crucial to collect a high-quality river dataset with flow velocity labels. The river datasets collected by existing methods are all based on images with a fixed resolution in monitoring devices, and then bound to the measured actual surface flow velocity values. Although this label data production method can correctly train the model in the deep learning manner, it is difficult to train a high-precision flow velocity recognition model due to the single scene and small number of labels of the original data.

[0053] Therefore, please refer to Figure 1 , Figure 1 which is a schematic flowchart of a method for enhancing river surface flow velocity recognition data provided in Embodiment 1 of the present invention. The method includes the following steps:

[0054] Step 11: Obtain a river dataset, where the river dataset includes the original river images and the flow velocity labels corresponding to the original river images.

[0055] In this step, the river dataset contains several original river images and the flow velocity labels corresponding to each original river image. The flow velocity labels are used to characterize the flow velocities of various points in the original river images.

[0056] Optionally, the river dataset can be sourced from the open-source Belgium dataset with flow velocity labels, i.e., the Belgium dataset, which is obtained by setting up a water tank in an indoor environment for simulation experiments and then taking pictures of the water flow images in the tank. Exemplarily, multiple water tanks can be set up, such as six. Therefore, six regions in the water flow images captured by screenshots can be used to achieve the purpose that the river states in each region are different, which is also convenient for making a river dataset for model training.

[0057] Step 12: Map the flow velocity labels to a grayscale value range to obtain a grayscale label map, where the grayscale value of any point in the grayscale label map corresponds to the flow velocity. Among them, the step of mapping the flow velocity labels to a grayscale value range to obtain a grayscale label map includes: if there is a point with a grayscale value of 0 after mapping the flow velocity labels to the grayscale value range, use the K-nearest neighbor algorithm to find the nearest neighboring point with a non-zero grayscale value, and use the neighboring point to fill the point with a grayscale value of 0.

[0058] Compared with the existing flow velocity scalar or flow velocity range, in this embodiment, by mapping the flow velocity labels to the grayscale value range of 0 to 255, a grayscale label map is made. That is to say, by converting the corresponding flow velocity label map in the original river image into a certain value within the grayscale value range of 0 to 255 and applying this grayscale value to the corresponding position in the original river image, a grayscale label map is obtained. Among them, the grayscale values at different positions represent different flow velocities.

[0059] In some embodiments, optionally, according to the characteristic that the flow velocity at both ends of the river channel is the lowest and the middle is the highest, normal distribution, Poisson distribution, etc. can be used to map the data to the 0-255 range.

[0060] In this embodiment, by normalizing the existing corresponding flow field distribution labels (i.e., flow velocity labels) of the original river image to the grayscale value range of 0 to 255, if there is a point with a value of 0 in the flow field, that is, a point with a corresponding grayscale value of 0, the K-nearest neighbor algorithm is used to find a nearest neighboring point with a non-zero grayscale value to this point with a value of 0, and use this neighboring point to fill the point with a grayscale value of 0. Through such a processing method, the abnormal flow velocity points in the image can be screened out, thereby ensuring the accuracy of the obtained training data.

[0061] Step 13: Randomly select regions of interest in each of the original river images and the grayscale label maps and perform cropping to obtain cropped images, and use the cropped images as training data for the deep learning model, where the positions and sizes of the regions of interest selected in the original river image and the corresponding grayscale label map are the same.

[0062] In this embodiment, by randomly selecting regions of interest (ROIs) in the original river image and the corresponding grayscale label image, and then cropping the regions of interest, cropped images are obtained. These cropped images can be used as the training data for the deep learning model, that is, using these cropped images as the input of the deep learning model, so that the deep learning model for identifying the flow velocity can identify the flow velocity values at different positions in the input image according to the input image, and then complete the training of the model.

[0063] In this step, the positions and sizes of the regions of interest selected in the original river image and the corresponding grayscale label image are the same, so as to ensure that the cropped images obtained by cropping in the original river image and the cropped images obtained by cropping in the grayscale label image correspond. Moreover, by randomly selecting regions of interest, different image receptive fields can be provided to improve the robustness of model training. Here, it should be noted that the so-called random selection of regions of interest may include that the position of the region of interest is random, or the size of the region of interest is random, or both the position and size of the region of interest are random.

[0064] Thus, the method for enhancing river surface flow velocity recognition data provided by the embodiment of the present invention maps the flow velocity labels to grayscale values to produce grayscale label images, generates corresponding flow velocity label values (i.e., grayscale values) for each pixel point of the original river image, and different label values in different regions improve the utilization efficiency of river data, making up for the scarcity of river data at the present stage. Compared with traditional flow velocity label images or optical flow images, it can improve the utilization efficiency of deep learning training data; at the same time, the method of random cropping is adopted to provide different image receptive fields, which can improve the robustness of model training.

[0065] The following is an example to illustrate the above method for enhancing river surface flow velocity recognition data.

[0066] In some embodiments of the present application, the step of randomly selecting regions of interest in each of the original river images and the grayscale label images and cropping them to obtain cropped images includes:

[0067] Randomly determine the size of the cropping frame corresponding to the region of interest, and the cropping frame is a square;

[0068] Obtain the width and height of the river channel region in the grayscale label image, and the position coordinates of the upper left corner and the lower right corner of the river channel region in the grayscale label image;

[0069] According to the size relationship between the width and height of the river channel region, determine the image type of the river channel region image, where the river channel region image is obtained by intercepting the river channel region from the grayscale label image, and the image type includes horizontal images and vertical images;

[0070] Determine the coordinate range of the upper left corner of the region of interest based on the image type, the width and height of the river channel region, the position coordinates of the upper left corner and the lower right corner of the river channel region in the grayscale label map, and the size of the cropping frame;

[0071] Crop the original river image and the grayscale label map according to the coordinate range of the upper left corner of the region of interest to obtain the cropped image.

[0072] In this embodiment, when randomly selecting the region of interest in the original river image and the grayscale label map, it is necessary to first randomly determine the size of the region of interest, that is, the cropping size, or rather, the size of the cropping frame corresponding to the region of interest. Exemplary cropping frame sizes can be 160×160, 320×320, etc.

[0073] Optionally, the shape of the cropping frame in this embodiment is a square, and the randomly determined size of the cropping frame can be randomly selected within a preset selection range. The minimum value of this preset selection range is generally set to 160, the maximum value is set to 640, and the random step size is set to 80. That is, the minimum size of the cropping frame can be 160×160, and the maximum can be 640×640. The random step size indicates that the size of the cropping frame of the original river image should be a value with an interval of 80 between 160 and 640, such as 240, 320, 400, 480, 560.

[0074] Further, by taking out the image corresponding to the river channel region in the grayscale label map, that is, the river channel region image, and then obtaining the width and height of the river channel region, and calculating the position of the river channel region in the grayscale label map. Among them, the position of the river channel region in the grayscale label map can be determined by the upper left corner coordinate position and the lower right corner coordinate position of the river channel region in the grayscale label map.

[0075] In this embodiment, take the upper left corner of the grayscale label map as the coordinate origin, and denote the abscissa and ordinate of the upper left corner of the river region, and the abscissa and ordinate of the lower right corner as .

[0076] In some embodiments, optionally, the determining the image type of the river channel region image according to the size relationship between the width and height of the river channel region includes:

[0077] If the width of the river channel region is greater than the height, determine that the image type is a landscape picture;

[0078] If the width of the river channel region is less than or equal to the height, determine that the image type is a portrait picture.

[0079] Specifically, when determining whether the image type of the river channel area image is a landscape image or a portrait image, if the width of the river channel area is greater than the height, it is a landscape image. At this time, the calculation formula for the selection range dh of the corresponding cropping area is dh = max(0, ), that is, the selection range in the height direction of the river channel area, generally is 40, which is an empirical value and is the preset height range value for the landscape image; if the width of the river channel area is less than or equal to the height, it is a portrait image. At this time, the calculation formula for the selection range dw of the corresponding cropping area is dw = max(0, ), that is, the selection range in the width direction of the river channel area, generally is 40, which is an empirical value and is the preset width range value for the portrait image.

[0080] In this embodiment, after determining the image type of the river channel area image, the coordinate range of the upper left corner of the region of interest can be determined according to the image type, the width and height of the river channel area, the position coordinates of the upper left corner and the lower right corner of the river channel area in the grayscale label image, and the size of the cropping frame. Then, a position is randomly selected within this coordinate range, and the size of the cropping frame is also randomly determined, so that the region of interest is finally determined. Then, the region of interest can be cropped at the corresponding positions in the original river image and the corresponding grayscale label image to obtain the cropped image.

[0081] In other embodiments, the step of determining the coordinate range of the upper left corner of the region of interest according to the image type, the width and height of the river channel area, the position coordinates of the upper left corner and the lower right corner of the river channel area in the grayscale label image, and the size of the cropping frame includes:

[0082] If the image type is a landscape image, the horizontal coordinate range of the upper left corner of the region of interest is [0, , and the vertical coordinate range is , ;

[0083] If the image type is a portrait image, the horizontal coordinate range of the upper left corner of the region of interest is , , and the vertical coordinate range is [0, ;

[0084] Wherein, w is the width of the river channel area, h is the height of the river channel area, l and t are respectively the horizontal coordinate and the vertical coordinate of the upper left corner of the river channel area in the grayscale label image, r and b are respectively the horizontal coordinate and the vertical coordinate of the lower right corner of the river channel area in the grayscale label image, is the width of the cropping frame, is the height of the cropping frame, is the preset height range value when the image type is a landscape image, is the preset width range value when the image type is a portrait image.

[0085] That is to say, if the river area image is a landscape image, the abscissa range of the upper left corner of the region of interest / cropping region is [0, , and the ordinate range is , , to ensure that the ordinate of the region of interest / cropping region is greater than dh, that is, to ensure that more images of the river area part can be cropped; similarly, if the river area image is a portrait image, the abscissa range of the upper left corner of the region of interest / cropping region is , , and the ordinate range is [0, , to ensure that the abscissa of the region of interest / cropping region is greater than dw, that is, to ensure that more images of the river area part can be cropped. Thus, more river information can be included in the finally cropped image, thereby improving the data utilization rate and the model training efficiency and accuracy.

[0086] In some embodiments, before the step of cropping the original river image and the grayscale label image according to the coordinate range of the upper left corner of the region of interest to obtain a cropped image, the method further includes:

[0087] If the size of the cropping frame is greater than the size of the river channel area, pixels with a grayscale value of 0 are used to perform boundary filling on the part of the region of interest in the original river image and the grayscale label image that exceeds the river channel area.

[0088] That is to say, when the size of the cropping frame / cropping region is greater than the size of the river channel area, it is necessary to perform boundary filling on the cropping region / region of interest in both the original river image and the grayscale label image. Specifically, for the part of the cropping frame that exceeds the river channel area, 0 pixels are used for filling, thereby creating an image filled with 0 pixels, that is, a pad image, and then covering this pad image to the corresponding position in the original river image / grayscale label image to obtain a boundary-filled image, and finally cropping the boundary-filled image to finally obtain a cropped image. This embodiment improves the robustness of the subsequent training model by performing boundary completion.

[0089] In the embodiments of the present invention, by mapping the flow velocity label to a grayscale value to produce a grayscale label image, compared with traditional flow velocity label images or optical flow images, the utilization efficiency of deep learning training data can be improved; at the same time, the method of randomly cropping is adopted to provide different image receptive fields, which can improve the robustness of model training.

[0090] The following combines with Figure 2Further introduce the process of randomly cropping images described above.

[0091] First, randomly determine the cropping size, that is, the cropping size is randomly determined to be 、 。

[0092] Then, extract the river channel area from the grayscale label map. Denote the width and height of the river channel area as w and h respectively.

[0093] Next, calculate the positions l, t, r, b of the river channel area in the grayscale label map, then w = r - l, h = b - t.

[0094] After that, judge whether b - t is greater than r - l. If so, it is a vertical picture, and the range of the upper - left corner coordinates of the cropping area is: the horizontal coordinate range is , , and the vertical coordinate range is [0, ; if not, it is a horizontal picture, and the range of the upper - left corner coordinates of the cropping area is: the horizontal coordinate range is [0, , and the vertical coordinate range is , .

[0095] Finally, judge whether the cropping area is larger than the size of the river channel area. If not, apply the above - mentioned upper - left corner coordinate range of the cropping area and the random cropping size to the original river picture and the grayscale label map to obtain and cropped images of the size, that is, including the cropped image obtained by cropping from the original river picture and the cropped map obtained by cropping from the grayscale label map; if so, perform boundary filling on the cropping area, that is, fill the part exceeding the river channel area with 0 pixels, and finally perform cropping.

[0096] Next, further introduce the method process from obtaining training data to model training.

[0097] First, use video surveillance equipment to collect river data, and at the same time obtain the flow velocity label corresponding to the river data, that is, obtain the original river picture and the flow velocity label corresponding to the original river picture. Then, use the unprocessed original river label data to remake the dataset. Specifically, according to the characteristic that the flow velocity at both ends of the river channel is the lowest and the middle is the highest, normal distribution, Poisson distribution, etc. can be used to map the data to the 0 - 255 interval to obtain the grayscale label map. After obtaining the original river picture and the grayscale label map, perform random region - of - interest cropping on the original river picture and the grayscale label map to obtain cropped images of different sizes. These cropped images are used as the training data of the deep - learning model, that is, as the input of the deep - learning model.

[0098] Optionally, the deep learning model adopts an improved version of the U-Net network model. To adapt to the input of multi-scale images, the number of convolutional kernels in the 5th layer of the U-Net is changed to 1, and a global average pooling layer is added to output a one-dimensional flow velocity scalar for the output of the flow velocity value. At the same time, the output result of the upsampling part is the flow velocity distribution of the original image, corresponding to the flow velocity distribution label value.

[0099] Furthermore, set the hyperparameters of the deep learning model, such as the number of iterations, learning rate, etc., and then input the training data obtained in the above steps into the deep learning model for model training.

[0100] Finally, taking the Belgium dataset as an example, verify the trained model with the test dataset. The output result of the final deep learning model is as Figure 3 shown. It can be seen that the training data made by using the river surface flow velocity recognition data enhancement method of this embodiment can enable the trained deep learning model to have a high flow velocity recognition accuracy.

[0101] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of a river surface flow velocity recognition data enhancement device provided in the second embodiment of the present invention. The device 40 includes:

[0102] An acquisition module 41, configured to acquire a river dataset, where the river dataset includes a river original image and a flow velocity label corresponding to the river original image;

[0103] A mapping module 42, configured to map the flow velocity label to a gray value interval to obtain a gray label map, where the gray value of any point in the gray label map corresponds to the flow velocity; the mapping module 42 includes a filling unit, and the filling unit is configured to, if there is a point with a gray value of 0 after mapping the flow velocity label to the gray value interval, use the K-nearest neighbor algorithm to find the nearest neighboring point with a gray value not equal to 0, and fill the point with a gray value of 0 using the neighboring point.

[0104] A cropping module 43, configured to randomly select a region of interest in each of the river original images and the gray label maps and perform cropping to obtain cropped images, and use the cropped images as training data for the deep learning model, where the positions and sizes of the regions of interest selected in the river original image and the corresponding gray label map are the same.

[0105] Optionally, the cropping module includes:

[0106] A first determination unit, configured to randomly determine the size of the cropping frame corresponding to the region of interest, and the cropping frame is a square;

[0107] An acquisition unit for acquiring the width and height of the river channel area in the grayscale label map, as well as the position coordinates of the upper left corner and the lower right corner of the river channel area in the grayscale label map;

[0108] A judgment unit for judging the image type of the river channel area image according to the size relationship between the width and height of the river channel area. The river channel area image is obtained by intercepting the river channel area from the grayscale label map, and the image types include horizontal pictures and vertical pictures;

[0109] A second determination unit for determining the coordinate range of the upper left corner of the region of interest according to the image type, the width and height of the river channel area, the position coordinates of the upper left corner and the lower right corner of the river channel area in the grayscale label map, and the size of the cropping frame;

[0110] A cropping unit for cropping the original river image and the grayscale label map according to the coordinate range of the upper left corner of the region of interest to obtain a cropped image.

[0111] Optionally, the second determination unit includes:

[0112] A first determination subunit for, if the image type is a horizontal picture, the horizontal coordinate range of the upper left corner of the region of interest is [0, , and the vertical coordinate range is , ;

[0113] A second determination subunit for, if the image type is a vertical picture, the horizontal coordinate range of the upper left corner of the region of interest is , , and the vertical coordinate range is [0, ;

[0114] Wherein, w is the width of the river channel area, h is the height of the river channel area, l and t are respectively the horizontal coordinate and the vertical coordinate of the upper left corner of the river channel area in the grayscale label map, r and b are respectively the horizontal coordinate and the vertical coordinate of the lower right corner of the river channel area in the grayscale label map, is the width of the cropping frame, is the height of the cropping frame, is the preset height range value when the image type is a horizontal picture, is the preset width range value when the image type is a vertical picture.

[0115] Optionally, the device further includes:

[0116] A filling module, configured to, if the size of the cropping frame is greater than the size of the river channel area, perform boundary filling on the part of the region of interest in the original river image and the grayscale label image that exceeds the river channel area with pixels having a grayscale value of 0.

[0117] Optionally, the determination unit includes:

[0118] A first determination subunit, configured to determine that the image type is a landscape image if the width of the river channel area is greater than the height;

[0119] A second determination subunit, configured to determine that the image type is a portrait image if the width of the river channel area is less than or equal to the height.

[0120] In the embodiment of the present invention, by mapping the flow velocity label to a grayscale value to produce a grayscale label image, the utilization efficiency of deep learning training data can be improved compared with traditional flow velocity label images or optical flow images; at the same time, a random cropping method is adopted to provide different image receptive fields, which can improve the robustness of model training.

[0121] The embodiment of the present invention is a product embodiment corresponding to the first method embodiment above, so it will not be repeated here. For details, please refer to the first embodiment above.

[0122] The above is the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A river surface velocity identification data enhancement method, characterized in that: The method comprises the following steps: Acquire a river data set, wherein the river data set includes an original river image and a flow rate label corresponding to the original river image; The flow rate label is mapped to a grayscale value interval to obtain a grayscale label map, wherein the grayscale value of any point in the grayscale label map corresponds to the flow rate, and the step of mapping the flow rate label to the grayscale value interval to obtain the grayscale label map comprises: if there is a point with a grayscale value of 0 after the flow rate label is mapped to the grayscale value interval, a K-nearest neighboring point whose grayscale value is not 0 is found by using a K-nearest neighboring point algorithm, and the point with a grayscale value of 0 is filled with the neighboring point; A region of interest is randomly selected from each of the river original image and the grayscale label image and cropped to obtain a cropped image, and the cropped image is used as training data for the deep learning model, wherein the position and size of the region of interest selected from the river original image and the corresponding grayscale label image are the same.

2. The method according to claim 1, characterized in that The step of randomly selecting an area of ​​interest from each of the river original images and the grayscale label image and cropping the area to obtain a cropped image comprises: Randomly determine the size of a cropping frame corresponding to the region of interest, where the cropping frame is a square; Obtain the width and height of the river channel area in the grayscale label image, and the position coordinates of the upper left corner and the lower right corner of the river channel area in the grayscale label image; According to the size relationship between the width and height of the river channel area, the image type of the river channel area image is determined, wherein the river channel area image is obtained by intercepting the river channel area from the grayscale label image, and the image type includes a horizontal image and a vertical image; Determine the coordinate range of the upper left corner of the region of interest according to the image type, the width and height of the river channel area, the position coordinates of the upper left corner and the lower right corner of the river channel area in the grayscale label image, and the size of the cropping frame; The original river image and the grayscale label image are cropped according to the coordinate range of the upper left corner of the region of interest to obtain a cropped image.

3. The method according to claim 2, characterized in that The step of determining the coordinate range of the upper left corner of the region of interest according to the image type, the width and height of the river channel area, the position coordinates of the upper left corner and the lower right corner of the river channel area in the grayscale label image, and the size of the cropping frame comprises: If the image type is a horizontal image, the horizontal coordinate range of the upper left corner of the region of interest is [0, ], the vertical axis range is [ , ]; If the image type is a vertical image, the horizontal coordinate range of the upper left corner of the region of interest is [ , ], the vertical axis range is [0, ]; Among them, w is the width of the river area, h is the height of the river area, l and t are the horizontal coordinate and vertical coordinate of the upper left corner of the river area in the grayscale label image, r and b are the horizontal coordinate and vertical coordinate of the lower right corner of the river area in the grayscale label image, respectively. is the width of the cropping frame, is the height of the cropping box, is the preset height range value when the image type is a horizontal image, It is the preset width range value when the image type is a vertical image.

4. The method according to claim 2, characterized in that: Before the step of cropping the original river image and the grayscale label image according to the coordinate range of the upper left corner of the region of interest to obtain a cropped image, the method further includes: If the size of the cropping frame is larger than the size of the river area, pixels with a grayscale value of 0 are used to fill the borders of the portion of the region of interest in the original river image and the grayscale label image that exceeds the river area.

5. The method according to claim 2, characterized in that: The determining the image type of the river channel region image according to the size relationship between the width and height of the river channel region comprises: If the width of the river channel area is greater than the height, determining that the image type is a horizontal image; If the width of the river channel area is less than or equal to the height, the image type is determined to be a vertical image.

6. A river surface velocity identification data enhancement device, characterized in that: include: An acquisition module, used for acquiring a river data set, wherein the river data set includes an original river image and a flow rate label corresponding to the original river image; A mapping module, used to map the flow rate label to a grayscale value interval to obtain a grayscale label map, wherein the grayscale value of any point in the grayscale label map corresponds to the flow rate, and the mapping module includes a filling unit, which is used to use a K-nearest neighbor algorithm to find the nearest neighboring point whose grayscale value is not 0 if there is a point with a grayscale value of 0 after mapping the flow rate label to the grayscale value interval, and use the neighboring point to fill the point with a grayscale value of 0; A cropping module is used to randomly select and crop a region of interest in each of the river original images and the grayscale label images to obtain a cropped image, and use the cropped image as training data for a deep learning model, wherein the position and size of the region of interest selected in the river original image and the corresponding grayscale label image are the same.

7. The device according to claim 6, characterized in that The cutting module comprises: A first determining unit, configured to randomly determine a size of a cropping frame corresponding to the region of interest, wherein the cropping frame is a square; An acquisition unit, used for acquiring the width and height of the river channel area in the grayscale label image, and the position coordinates of the upper left corner and the lower right corner of the river channel area in the grayscale label image; A judging unit, configured to judge the image type of the river channel region image according to the size relationship between the width and height of the river channel region, wherein the river channel region image is obtained by intercepting the river channel region from the grayscale label image, and the image type includes a horizontal image and a vertical image; A second determining unit is used to determine the coordinate range of the upper left corner of the region of interest according to the image type, the width and height of the river channel area, the position coordinates of the upper left corner and the lower right corner of the river channel area in the grayscale label image, and the size of the cropping frame; A cropping unit is used to crop the original river image and the grayscale label image according to the coordinate range of the upper left corner of the region of interest to obtain a cropped image.

8. The device according to claim 7, characterized in that The second determining unit includes: The first determining subunit is configured to determine that if the image type is a horizontal image, the horizontal coordinate range of the upper left corner of the region of interest is [0, ], the vertical axis range is [ , ]; The second determining subunit is used for determining that if the image type is a vertical image, the horizontal coordinate range of the upper left corner of the region of interest is [ , ], the vertical axis range is [0, ]; Among them, w is the width of the river area, h is the height of the river area, l and t are the horizontal coordinate and vertical coordinate of the upper left corner of the river area in the grayscale label image, r and b are the horizontal coordinate and vertical coordinate of the lower right corner of the river area in the grayscale label image, respectively. is the width of the cropping frame, is the height of the cropping box, is the preset height range value when the image type is a horizontal image, It is the preset width range value when the image type is a vertical image.

Images