A method and device for measuring underwater sediment movement elements based on deep learning

Abstract

The present invention provides a method and device for measuring underwater sediment movement elements based on deep learning, which relate to the technical field of underwater sediment measurement. The specific implementation scheme is: obtaining first underwater sediment movement image information; correcting the first underwater sediment movement image information based on an image distortion correction algorithm to obtain the first underwater sediment movement image information after correction; determining first image scale information according to the underwater sediment movement element measurement device based on deep learning and the first underwater sediment movement image information after correction; identifying and tracking the first underwater sediment movement image information after correction based on a preset deep learning multi-target tracking model to obtain first sediment particle identification and tracking results; performing data analysis on the first sediment particle identification and tracking results based on the first image scale information to obtain first sediment movement element information, thereby improving the monitoring efficiency and accuracy of underwater sediment movement element information.

Description

Technical Field

[0001] The present disclosure relates to the technical field of underwater sediment measurement, and in particular to a method and device for measuring underwater sediment movement elements based on deep learning. Background Art

[0002] With the development of water conservancy engineering technology, underwater sediment measurement technology has become increasingly important. Underwater sediment measurement technology is mainly used in water conservancy projects, waterway maintenance, riverbed evolution and environmental change monitoring and research. In related technologies, the movement element information of underwater sediment is measured using artificial single-point measurement methods. Summary of the invention

[0003] The present invention provides a method and device for measuring underwater sediment movement elements based on deep learning.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for measuring underwater sediment movement elements based on deep learning is provided, comprising:

[0005] Acquire first underwater sediment movement image information; the first underwater sediment movement image information is obtained by shooting in a video mode using an underwater camera module in an underwater sediment movement element measurement device based on deep learning;

[0006] Correcting the first underwater sediment motion image information based on an image distortion correction algorithm to obtain the corrected first underwater sediment motion image information;

[0007] Determine the first image scale information according to the side length information of the square bottom surface of the rectangular parallelepiped frame in the underwater sediment movement element measurement device based on deep learning and the first underwater sediment movement image information after correction processing; the first image scale information includes the riverbed surface size information corresponding to each pixel in the first underwater sediment movement image information after correction processing;

[0008] Based on a preset deep learning multi-target tracking model, the first underwater sediment motion image information after the correction is identified and tracked to obtain a first sediment particle identification and tracking result; the preset deep learning multi-target tracking model takes the underwater sediment motion image as input and takes the sediment particle identification and tracking result as output;

[0009] Based on the first image scale information, data analysis is performed on the first sediment particle identification and tracking result to obtain first sediment movement element information.

[0010] According to a second aspect of an embodiment of the present disclosure, there is provided an underwater sediment movement element measurement device based on deep learning, comprising:

[0011] A rectangular frame used to fix the position of the underwater sediment movement element measurement device based on deep learning;

[0012] A camera connection module is connected to the top of the rectangular frame and is used to fix the underwater camera module on the rectangular frame;

[0013] An underwater camera module, connected to the camera connection module, for photographing underwater sediment at a location where a deep learning-based underwater sediment movement element measurement device is located;

[0014] The processing module is configured to execute the underwater sediment movement element measurement method based on deep learning described in the first aspect above.

[0015] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0016] at least one processor; and

[0017] a memory communicatively connected to at least one processor; wherein,

[0018] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute the method of the first aspect mentioned above.

[0019] According to a fourth aspect of an embodiment of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method of the first aspect.

[0020] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, which implements the steps of any one of the methods in the first aspect when executed by a processor.

[0021] The technical solution disclosed in the present invention obtains first underwater sediment motion image information; the first underwater sediment motion image information is obtained by shooting in a camera mode using an underwater camera module in an underwater sediment motion element measurement device based on deep learning; the first underwater sediment motion image information is corrected based on an image distortion correction algorithm to obtain the first underwater sediment motion image information after correction; the first picture scale information is determined according to the side length information of the square bottom surface of the rectangular parallelepiped frame in the underwater sediment motion element measurement device based on deep learning and the first underwater sediment motion image information after correction; the first picture scale information includes the first underwater sediment after correction The riverbed size information corresponding to each pixel in the motion image information; based on the preset deep learning multi-target tracking model, the first underwater sediment motion image information after correction is identified and tracked to obtain the first sediment particle identification and tracking result; the preset deep learning multi-target tracking model takes the underwater sediment motion image as input and the sediment particle identification and tracking result as output; based on the first image scale information, the first sediment particle identification and tracking result is analyzed to obtain the first sediment motion element information, which can improve the monitoring efficiency and accuracy of the underwater sediment motion element information, so as to provide reliable data support for research and application in related fields.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0024] Figure 1 It is a flow chart of a method for measuring underwater sediment movement elements based on deep learning according to an embodiment of the present disclosure;

[0025] Figure 2 It is a flow chart of a method for measuring underwater sediment movement elements based on deep learning according to an embodiment of the present disclosure;

[0026] Figure 3 It is a flow chart of a method for measuring underwater sediment movement elements based on deep learning according to an embodiment of the present disclosure;

[0027] Figure 4 is a block diagram of an underwater sediment movement element measurement device based on deep learning according to an embodiment of the present disclosure;

[0028] Figure 5 is a block diagram of an underwater sediment movement element measurement device based on deep learning according to an embodiment of the present disclosure;

[0029] Figure 6 is a block diagram of an underwater sediment movement element measurement device based on deep learning according to an embodiment of the present disclosure;

[0030] Figure 7 is a block diagram of an electronic device provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0032] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0033] Figure 1 It is a flow chart of a method for measuring underwater sediment movement elements based on deep learning according to an embodiment of the present disclosure. It should be noted that, in some embodiments, the execution subject of the method for measuring underwater sediment movement elements based on deep learning can be a processing module of the device for measuring underwater sediment movement elements based on deep learning provided in an embodiment of the present disclosure. Figure 1 As shown, the underwater sediment movement element measurement method based on deep learning includes but is not limited to the following steps:

[0034] In step S101, first underwater sediment motion image information is obtained.

[0035] It should be noted that, in some embodiments, the first underwater sediment movement image information is obtained by photographing in a video mode using an underwater camera module in an underwater sediment movement element measurement device based on deep learning.

[0036] In step S102, the first underwater sediment motion image information is corrected based on an image distortion correction algorithm to obtain the corrected first underwater sediment motion image information.

[0037] It should be noted that, in some embodiments, the first underwater sediment motion image information is corrected according to the coordinate information of the rectangular frame in the underwater sediment motion element measurement device based on deep learning to obtain the corrected first underwater sediment motion image information.

[0038] In step S103, first image scale information is determined according to the side length information of the square bottom surface of the rectangular parallelepiped frame in the underwater sediment movement element measurement device based on deep learning and the first underwater sediment movement image information after correction processing.

[0039] It should be noted that, in some embodiments, the spatial resolution is the actual ground distance represented by one pixel in the photo. Optionally, in some embodiments, the first image scale information includes the riverbed size information corresponding to each pixel in the first underwater sediment motion image information after correction processing. Exemplarily, the first image scale information is five centimeters, that is, each pixel in the first underwater sediment motion image information after correction processing corresponds to a square riverbed with a side length of five centimeters.

[0040] In step S104, the first underwater sediment motion image information after correction is identified and tracked based on a preset deep learning multi-target tracking model to obtain a first sediment particle identification and tracking result.

[0041] It should be noted that, in some embodiments, the preset deep learning multi-target tracking model uses underwater sediment motion images as input and sediment particle recognition and tracking results as output. Optionally, in some embodiments, the preset deep learning multi-target tracking model is a multi-target tracking model using the DeepSORT algorithm as a tracker and the YOLOv8 algorithm as a detector.

[0042] In step S105, data analysis is performed on the first sediment particle identification and tracking result based on the first image scale information to obtain first sediment movement element information.

[0043] It should be noted that, in some embodiments, the first sediment movement element information includes sediment movement velocity information and cross-sectional sediment flux information. Optionally, in some embodiments, data analysis is performed on the first sediment particle identification and tracking result based on the first image scale information to obtain sediment movement velocity information and cross-sectional sediment flux information.

[0044] In an embodiment of the present disclosure, first underwater sediment motion image information is obtained; the first underwater sediment motion image information is obtained by shooting in a camera mode using an underwater camera module in an underwater sediment motion element measurement device based on deep learning; the first underwater sediment motion image information is corrected based on an image distortion correction algorithm to obtain the corrected first underwater sediment motion image information; first picture scale information is determined based on the side length information of the square bottom surface of the rectangular parallelepiped frame in the underwater sediment motion element measurement device based on deep learning and the corrected first underwater sediment motion image information; the first picture scale information includes the first underwater sediment motion image information after the correction. The riverbed size information corresponding to each pixel in the sand movement image information; based on the preset deep learning multi-target tracking model, the first underwater sediment movement image information after correction is identified and tracked to obtain the first sediment particle identification and tracking result; the preset deep learning multi-target tracking model takes the underwater sediment movement image as input and the sediment particle identification and tracking result as output; based on the first image scale information, the first sediment particle identification and tracking result is analyzed to obtain the first sediment movement element information, which can improve the monitoring efficiency and accuracy of the underwater sediment movement element information, so as to provide reliable data support for research and application in related fields.

[0045] Figure 2 1 is a flow chart of a method for measuring underwater sediment movement elements based on deep learning according to an embodiment of the present disclosure. Figure 2 As shown, the underwater sediment movement element measurement method based on deep learning includes but is not limited to the following steps:

[0046] In step S201, first underwater sediment motion image information is obtained.

[0047] In the embodiments of the present disclosure, step S201 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0048] In step S202, a first direct linear transformation formula group is constructed.

[0049] Optionally, in some embodiments, a first direct linear conversion formula group is constructed according to the first world coordinate parameter, the first image coordinate parameter and the first distortion parameter. Exemplarily, the four corners of the square bottom surface of the rectangular parallelepiped frame in the underwater sediment movement element measurement device based on deep learning determine the first world coordinate parameter, the first image coordinate parameter and the first distortion parameter; and a first direct linear conversion formula group is constructed according to the first world coordinate parameter, the first image coordinate parameter and the first distortion parameter. For example, the first direct linear conversion formula group can be expressed by the following formula (1):

[0050]

[0051] Among them, x i Refers to the first parameter of the i-th corner in the first image coordinate parameter; y i Refers to the second parameter of the i-th corner in the first image coordinate parameter; X i Refers to the first parameter of the i-th corner in the first world coordinate parameter; Y i Refers to the second parameter of the i-th corner in the first world coordinate parameter; a i1 refers to the first parameter of the i-th corner in the first distortion parameter; a i2 refers to the second parameter of the i-th corner in the first distortion parameter; a i3 refers to the third parameter of the i-th corner in the first distortion parameter; a i4 refers to the fourth parameter of the i-th corner in the first distortion parameter; a i5 refers to the fifth parameter of the i-th corner in the first distortion parameter; a i6 refers to the sixth parameter of the i-th corner in the first distortion parameter; a i7 refers to the seventh parameter of the i-th corner in the first distortion parameter; a i8 Refers to the eighth parameter of the i-th corner in the first distortion parameter.

[0052] In step S203, first world coordinate information and first image coordinate information are obtained.

[0053] It should be noted that, in some embodiments, the first world coordinate information is the world coordinate information of the four corners of the square bottom surface of the rectangular frame in the underwater sediment movement element measurement device based on deep learning; the first image coordinate information is the image coordinate information of the four corners of the square bottom surface of the rectangular frame in the underwater sediment movement element measurement device based on deep learning.

[0054] In step S204, a first direct linear transformation formula group is solved based on the first world coordinate information and the first image coordinate information to obtain a first distortion parameter.

[0055] Optionally, in some embodiments, the first world coordinate information and the first image coordinate information are substituted into a first direct linear conversion formula group for solution to obtain a first distortion parameter.

[0056] In step S205, the first underwater sediment motion image information is corrected based on the first distortion parameter to obtain the corrected first underwater sediment motion image information.

[0057] Optionally, in some embodiments, first blank image information is created; each pixel in the first underwater sediment motion image information is corrected based on the first distortion parameter, and each corrected pixel is mapped to the first blank image information to obtain the corrected first underwater sediment motion image information.

[0058] In step S206, the first image scale information is determined according to the side length information of the square bottom surface of the rectangular parallelepiped frame in the underwater sediment movement element measurement device based on deep learning and the first underwater sediment movement image information after correction processing.

[0059] In the embodiments of the present disclosure, step S206 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0060] In step S207, the first underwater sediment motion image information after the correction is identified and tracked based on the preset deep learning multi-target tracking model to obtain a first sediment particle identification and tracking result.

[0061] In the embodiments of the present disclosure, step S207 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0062] In step S208, data analysis is performed on the first sediment particle identification and tracking result based on the first image scale information to obtain first sediment movement element information.

[0063] In the embodiments of the present disclosure, step S208 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0064] In an embodiment of the present disclosure, a first direct linear conversion formula group is constructed; first world coordinate information and first image coordinate information are obtained; the first direct linear conversion formula group is solved based on the first world coordinate information and the first image coordinate information to obtain a first distortion parameter; the first underwater sediment motion image information is corrected based on the first distortion parameter to obtain the corrected first underwater sediment motion image information, which can further improve the accuracy of the underwater sediment motion element information.

[0065] Figure 3 1 is a flow chart of a method for measuring underwater sediment movement elements based on deep learning according to an embodiment of the present disclosure. Figure 3 As shown, the underwater sediment movement element measurement method based on deep learning includes but is not limited to the following steps:

[0066] In step S301, first underwater sediment motion image information is obtained.

[0067] In the embodiments of the present disclosure, step S301 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0068] In step S302, the first underwater sediment motion image information is corrected based on an image distortion correction algorithm to obtain the corrected first underwater sediment motion image information.

[0069] In the embodiments of the present disclosure, step S302 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0070] In step S303, first image scale information is determined according to the side length information of the square bottom surface of the rectangular parallelepiped frame in the underwater sediment movement element measurement device based on deep learning and the first underwater sediment movement image information after correction processing.

[0071] In the embodiments of the present disclosure, step S303 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0072] In step S304, the first underwater sediment motion image information after correction is identified and tracked based on a preset deep learning multi-target tracking model to obtain a first sediment particle identification and tracking result.

[0073] In the embodiments of the present disclosure, step S304 may be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0074] In step S305, the first sediment particle recognition and tracking result is scale-transformed based on the first image scale information to obtain the scale-transformed first sediment particle recognition and tracking result.

[0075] Optionally, in some embodiments, the first sediment particle identification and tracking result includes but is not limited to tracking image information of each first sediment particle among the plurality of first sediment particles.

[0076] Optionally, in some embodiments, the tracking image information of each first silt particle in the plurality of first silt particles is scale-transformed based on the first image scale information to obtain the scale-transformed first silt particle identification and tracking result. Exemplarily, the tracking image information of each first silt particle in the plurality of first silt particles includes displacement information; the displacement information of any first silt particle in the plurality of first silt particles is 5 pixels; the first image scale information is 5 centimeters; the tracking image information of each first silt particle in the plurality of first silt particles is scale-transformed based on the first image scale information, and the displacement information of the first silt particle is 25 centimeters.

[0077] In step S306, data analysis is performed on the scale-transformed first sediment particle identification and tracking result to obtain first sediment movement element information.

[0078] Optionally, in some embodiments, the first sediment movement element information includes sediment movement velocity information and cross-sectional sediment flux information.

[0079] It should be noted that, in some embodiments, the scale-transformed first sediment particle identification and tracking result includes but is not limited to the displacement information of each first sediment particle in the plurality of first sediment particles. Optionally, in some embodiments, the first average displacement information is determined based on the displacement information of the plurality of first sediment particles; the first frame interval information is determined based on the start frame and the end frame of the first underwater sediment motion image information; the first average displacement information is divided by the first frame interval information to obtain sediment motion speed information.

[0080] It should be noted that, in some embodiments, the cross-section sediment flux information is the sediment mass information passing through the first section per unit time in the first underwater sediment movement image information.

[0081] It should be noted that, in some embodiments, the scale-transformed first sediment particle identification and tracking results include but are not limited to the mass information of each first sediment particle among the multiple first sediment particles. Optionally, in some embodiments, the area information of the first section is determined; based on the scale-transformed first sediment particle identification and tracking results, multiple second sediment particles passing through the first section are determined; for each second sediment particle among the multiple second sediment particles, the displacement information of the second sediment particle is divided by the first frame interval information to obtain the velocity information of the second sediment particle; based on the area information of the first section, the mass information and velocity information of the multiple second sediment particles, the sediment flux information of the section is determined.

[0082] For example, the sediment flux information of the section can be calculated by the following formula (2):

[0083]

[0084] Where Q refers to the cross-section sediment flux information; N refers to the number of multiple second sediment particles; v i refers to the velocity information of the second sediment particle of the ith order; m i refers to the area information of the second ith sediment particle; A refers to the area information of the first section.

[0085] In an embodiment of the present disclosure, a scale transformation is performed on the first sediment particle identification and tracking result based on the first image scale information to obtain the first sediment particle identification and tracking result after the scale transformation; data analysis is performed on the first sediment particle identification and tracking result after the scale transformation to obtain the first sediment movement element information, which can further improve the accuracy of the underwater sediment movement element information.

[0086] Figure 4 is a block diagram of an underwater sediment movement element measurement device based on deep learning according to an embodiment of the present disclosure. Figure 4 As shown, the underwater sediment movement element measurement device includes but is not limited to an acquisition module 401 , a correction module 402 , a determination module 403 , an identification and tracking module 404 , and a data analysis module 405 .

[0087] The acquisition module 401 is used to acquire first underwater sediment motion image information; the first underwater sediment motion image information is obtained by shooting in a camera mode using an underwater camera module in an underwater sediment motion element measurement device based on deep learning;

[0088] The correction module 402 is used to correct the first underwater sediment motion image information based on an image distortion correction algorithm to obtain the corrected first underwater sediment motion image information;

[0089] The determination module 403 is used to determine the first image scale information according to the side length information of the square bottom surface of the rectangular parallelepiped frame in the underwater sediment movement element measurement device based on deep learning and the first underwater sediment movement image information after correction processing; the first image scale information includes the riverbed size information corresponding to each pixel in the first underwater sediment movement image information after correction processing;

[0090] The recognition and tracking module 404 is used to perform recognition and tracking processing on the first underwater sand motion image information after the correction processing based on the preset deep learning multi-target tracking model to obtain the first sand particle recognition and tracking result; the preset deep learning multi-target tracking model takes the underwater sand motion image as input and takes the sand particle recognition and tracking result as output;

[0091] The data analysis module 405 is used to perform data analysis on the first sediment particle identification and tracking result based on the first image scale information to obtain first sediment movement element information.

[0092] As an example, the correction module 402 is also used to construct a first direct linear conversion formula group; obtain first world coordinate information and first image coordinate information; the first world coordinate information is the world coordinate information of the four corners of the square bottom surface of the rectangular frame in the underwater sediment movement element measurement device based on deep learning; the first image coordinate information is the image coordinate information of the four corners of the square bottom surface of the rectangular frame in the underwater sediment movement element measurement device based on deep learning; the first direct linear conversion formula group is solved based on the first world coordinate information and the first image coordinate information to obtain a first distortion parameter; the first underwater sediment movement image information is corrected based on the first distortion parameter to obtain the corrected first underwater sediment movement image information.

[0093] As an example, the data analysis module 405 is also used to scale the first sediment particle identification and tracking result based on the first image scale information to obtain the first sediment particle identification and tracking result after scale transformation; perform data analysis on the first sediment particle identification and tracking result after scale transformation to obtain the first sediment movement element information.

[0094] As an example, the deep learning multi-target tracking model preset in the data analysis module 405 is a multi-target tracking model using the DeepSORT algorithm as a tracker and the YOLOv8 algorithm as a detector.

[0095] As an example, the first sediment movement element information in the data analysis module 405 includes sediment movement velocity information and cross-sectional sediment flux information.

[0096] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0097] Figure 5 is a block diagram of an underwater sediment movement element measurement device based on deep learning according to an embodiment of the present disclosure. Figure 5 As shown, the underwater sediment movement element measurement device based on deep learning includes but is not limited to a rectangular frame 510, a camera connection module 520, an underwater camera module 530, and a processing module 540.

[0098] The rectangular frame 510 is used to fix the position of the underwater sediment movement element measurement device based on deep learning;

[0099] A camera connection module 520 is connected to the rectangular parallelepiped frame and is used to fix the underwater camera module on the rectangular parallelepiped frame;

[0100] An underwater camera module 530 is connected to the camera connection module and is used to photograph the underwater sediment at the location of the underwater sediment movement element measurement device based on deep learning;

[0101] The processing module 540 is configured to execute the underwater sediment movement element measurement method based on deep learning provided in the embodiment of the present disclosure.

[0102] Optionally, in some embodiments, the bottom surface of the rectangular parallelepiped frame 510 is a square. Exemplarily, the first image scale information is determined according to the side length information of the square bottom surface of the rectangular parallelepiped frame 510 in the underwater sediment movement element measurement device based on deep learning and the first underwater sediment movement image information after correction processing.

[0103] Optionally, in some embodiments, the size information of the rectangular parallelepiped frame 510 is determined according to the water depth information of the location of the underwater sediment movement element measurement device based on deep learning. Exemplarily, the water depth information of the location of the underwater sediment movement element measurement device based on deep learning is 40 centimeters; the height information of the rectangular parallelepiped frame 510 can be 50 centimeters.

[0104] Optionally, in some embodiments, the underwater camera module 630 in the underwater sediment movement element measurement device based on deep learning adopts a video recording mode to obtain the first underwater sediment movement image information.

[0105] In some embodiments, the underwater camera module 530 in the underwater sediment movement element measurement device based on deep learning in the embodiments of the present disclosure also includes: an underwater camera 531, a waterproof housing 532, and an underwater fill light array 533.

[0106] The underwater camera 531 is used to photograph the riverbed sediment at the location of the underwater sediment movement element measurement device based on deep learning;

[0107] A waterproof housing 532, used to provide waterproof protection for the underwater camera;

[0108] The underwater fill light array 533 is used to provide lighting for the underwater camera 531 to improve the imaging quality of the photo.

[0109] In some embodiments, Figure 6 As shown, in Figure 5 On the basis of what is shown, the camera connection module 520 in the underwater sediment movement element measurement device based on deep learning in the embodiment of the present disclosure also includes: a transverse connecting rod 521, an adjustable connecting piece 522, and a retractable camera fixing rod 523.

[0110] Wherein, the transverse connecting rod 521 is connected to the top of the rectangular parallelepiped frame 510;

[0111] An adjustable connecting member 522 connected to the transverse connecting rod 521 and used to adjust the shooting angle of the underwater camera module 530;

[0112] The retractable camera fixing rod 523 is connected to the adjustable connecting piece 522 and the underwater camera module 530. According to the water depth information of the location of the underwater sediment movement element measuring device based on deep learning, the upper and lower positions of the underwater camera module 530 are adjusted to ensure that the underwater camera module 530 is in an underwater environment.

[0113] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a storage medium, and a computer program product.

[0114] Figure 7 1 is a block diagram of an electronic device provided according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices.

[0115] Optionally, in some embodiments, the electronic device may be a processing module in the underwater sediment movement element measurement device based on deep learning provided in the embodiments of the present disclosure.

[0116] The components shown herein, their connections and relationships, and their functions, are intended only as examples, and are not meant to limit implementations of the disclosure described and / or claimed herein.

[0117] like Figure 7 As shown, the electronic device includes: one or more processors 701, a memory 702, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 701 is taken as an example.

[0118] The memory 702 is a non-transient computer-readable storage medium provided by the present disclosure. The memory stores instructions executable by at least one processor, so that the at least one processor executes the underwater sediment movement element measurement method based on deep learning provided by the present disclosure. The non-transient computer-readable storage medium of the present disclosure stores computer instructions, which are used to cause a computer to execute the underwater sediment movement element measurement method based on deep learning provided by the present disclosure.

[0119] The memory 702 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the underwater sediment movement element measurement method based on deep learning in the embodiment of the present disclosure (for example, the attached Figure 4 The processor 701 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 702, that is, the underwater sediment movement element measurement method based on deep learning in the above method embodiment is implemented.

[0120] The memory 702 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 702 may optionally include a memory remotely arranged relative to the processor 701, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0121] The electronic device may further include: an input device 703 and an output device 704. The processor 701, the memory 702, the input device 703 and the output device 704 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0122] The input device 703 can receive input digital or character information, and generate key signal input related to the user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 704 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0123] Optionally, in some embodiments, a computer program product is also provided, including a computer program, which, when executed by one or more processors 701, can implement the underwater sediment movement element measurement method based on deep learning provided in the embodiments of the present disclosure.

[0124] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0126] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0127] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0128] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0129] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0130] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for measuring underwater sediment movement elements based on deep learning, characterized in that: The following steps are involved: Acquire first underwater sediment movement image information; the first underwater sediment movement image information is obtained by shooting in a video mode using an underwater camera module in an underwater sediment movement element measurement device based on deep learning; Correcting the first underwater sediment motion image information based on an image distortion correction algorithm to obtain the corrected first underwater sediment motion image information; Determine first image scale information according to the side length information of the square bottom surface of the rectangular parallelepiped frame in the underwater sediment movement element measurement device based on deep learning and the first underwater sediment movement image information after the correction processing; the first image scale information includes the riverbed surface size information corresponding to each pixel in the first underwater sediment movement image information after the correction processing; Based on a preset deep learning multi-target tracking model, the first underwater sediment motion image information after the correction is identified and tracked to obtain a first sediment particle identification and tracking result; the preset deep learning multi-target tracking model takes the underwater sediment motion image as input and takes the sediment particle identification and tracking result as output; Based on the first image scale information, data analysis is performed on the first sediment particle identification and tracking result to obtain first sediment movement element information.

2. The method according to claim 1, characterized in that The correcting the first underwater sediment motion image information based on the image distortion correction algorithm to obtain the corrected first underwater sediment motion image information includes: Constructing a first direct linear transformation formula group; Acquire first world coordinate information and first image coordinate information; the first world coordinate information is the world coordinate information of the four corners of the square bottom surface of the rectangular parallelepiped frame in the underwater sediment movement element measurement device based on deep learning; the first image coordinate information is the image coordinate information of the four corners of the square bottom surface of the rectangular parallelepiped frame in the underwater sediment movement element measurement device based on deep learning; Solving the first direct linear transformation formula group based on the first world coordinate information and the first image coordinate information to obtain a first distortion parameter; The first underwater sediment motion image information is corrected based on the first distortion parameter to obtain the corrected first underwater sediment motion image information.

3. The method according to claim 1, characterized in that The performing of data analysis on the first sediment particle identification and tracking result based on the first image scale information to obtain first sediment movement element information includes: Performing a scale transformation on the first sediment particle identification and tracking result based on the first image scale information to obtain a scale-transformed first sediment particle identification and tracking result; Data analysis is performed on the scale-transformed first sediment particle identification and tracking result to obtain the first sediment movement element information.

4. The method according to claim 1, characterized in that: The first sediment movement element information includes sediment movement velocity information and cross-sectional sediment flux information.

5. The method according to claim 1, characterized in that The preset deep learning multi-target tracking model is a multi-target tracking model using the DeepSORT algorithm as a tracker and the YOLOv8 algorithm as a detector.

6. The method according to any one of claims 1 to 5, characterized in that: The underwater sediment movement element measurement device based on deep learning includes: A rectangular parallelepiped frame, used to fix the position of the underwater sediment movement element measurement device based on deep learning; A camera connection module, connected to the rectangular parallelepiped frame, and used to fix the underwater camera module on the rectangular parallelepiped frame; The underwater camera module is connected to the camera connection module and is used to photograph the underwater sediment at the location where the deep learning-based underwater sediment movement element measurement device is located.

7. The method according to claim 6, characterized in that The underwater camera module comprises: An underwater camera, used to photograph the riverbed sediment at the location where the deep learning-based underwater sediment movement element measurement device is located; A waterproof housing, used to provide waterproof protection for the underwater camera; Underwater fill light array.

8. The method according to claim 6, characterized in that The underwater camera module comprises: A transverse connecting rod connected to the top of the rectangular parallelepiped frame; An adjustable connecting piece, connected to the transverse connecting rod, for adjusting the shooting angle of the underwater camera module; A retractable camera fixing rod is connected to the adjustable connecting piece and the underwater camera module, and adjusts the upper and lower positions of the underwater camera module according to the water depth information of the location of the underwater sediment movement element measuring device based on deep learning to ensure that the underwater camera module is in an underwater environment.

9. A device for measuring underwater sediment movement elements based on deep learning, characterized in that: include: A rectangular parallelepiped frame, used to fix the position of the underwater sediment movement element measurement device based on deep learning; A camera connection module connected to the top of the rectangular frame and used to fix the underwater camera module on the rectangular frame; The underwater camera module is connected to the camera connection module and is used to photograph the underwater sediment at the location where the underwater sediment movement element measurement device based on deep learning is located; The processing module is configured to execute the underwater sediment movement element measurement method based on deep learning as described in any one of claims 1-8.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images