Underwater video robot data acquisition method and device

By using CNN in underwater robots for target detection and comprehensive environmental scoring, and dynamically adjusting the data transmission strategy, the problem of low data acquisition efficiency of underwater robots in complex environments is solved, and efficient and accurate data acquisition and transmission are achieved.

CN119516441AActive Publication Date: 2025-02-25湖北亿立能科技股份有限公司
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411667715.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-21
Publication Date
2025-02-25
Estimated Expiration
2044-11-21

AI Technical Summary

Technical Problem

Existing underwater robots have problems such as low efficiency, high risk and difficulty in processing large amounts of data in real time in data collection and processing. Especially in complex underwater environments, factors such as insufficient light, strong fluidity and changes in water turbidity affect sensor performance, reducing the accuracy of data collection.

Method used

Convolutional neural network (CNN) is used for object detection, combined with a variety of environmental data (turbidity, temperature, salinity) to calculate the comprehensive environmental score, dynamically adjust the data transmission strategy, and priority is given to the transmission of important data.

Benefits of technology

Real-time identification and classification of underwater organisms and obstacles in complex underwater environments is achieved, the efficiency and accuracy of data collection is improved, data transmission strategies are optimized, and critical data transmission is ensured in a timely manner.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119516441A_ABST
    Figure CN119516441A_ABST
Patent Text Reader

Abstract

The invention discloses an underwater video robot data acquisition method and device, and relates to the technical field of video monitoring processing equipment, and the method comprises the steps: enabling a video acquisition device and a plurality of sensors to be installed on an underwater video robot; collecting video data, environment data and corresponding time information in real time; carrying out target detection on the video data by adopting a convolutional neural network (CNN), and calculating a first confidence coefficient of a target detection result; calculating a first comprehensive environment score at the current moment based on the environment data; calculating the data priority of the current moment based on the first confidence coefficient and the first comprehensive environment score; and transmitting the corresponding data to a ground control station based on the data priority. The method has the following advantages of real-time target detection and classification, improvement of data acquisition efficiency, dynamic adjustment of weight coefficients, adaptability of data transmission and the like.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of video monitoring processing equipment, and in particular relates to an underwater video robot data acquisition method and device. Background Art

[0002] With the development of underwater detection technology, underwater robots are increasingly used in fields such as marine research, environmental monitoring, resource exploration, and underwater construction. These fields have put forward higher requirements on the performance and intelligence of underwater robots, especially in data collection and processing. Traditional underwater data collection methods mostly rely on manual operation, which is inefficient, risky, and difficult to process large amounts of data in real time. Therefore, developing an underwater robot that can automatically and efficiently collect data has become a current research hotspot.

[0003] Underwater robots are usually equipped with a variety of sensors, such as high-definition cameras, sonar, temperature sensors, salinity sensors, and turbidity sensors. These sensors can provide information about the underwater environment. However, how to effectively integrate data from different sensors and process and analyze them in real time to make quick decisions remains a technical challenge. In a complex underwater environment, factors such as insufficient light, strong fluidity, and changes in water turbidity can affect the performance of sensors, thereby reducing the accuracy of data collection.

[0004] When underwater robots transmit data, they must consider network bandwidth and latency issues. Transmitting large amounts of data in real time may cause information overload, which will affect the response time of the system. Therefore, how to optimize the data transmission strategy based on the importance and priority of the data has become one of the keys to improving the efficiency of underwater data collection. Summary of the invention

[0005] In view of the defects in the above-mentioned prior art, the present invention provides a method for collecting data of an underwater video robot, comprising the following steps: Step S101, installing a video acquisition device and a plurality of sensors on the underwater video robot; Step S103, real-time acquisition of video data and environmental data, as well as corresponding time information, wherein the environmental data includes turbidity data, temperature and salinity; Step S105: performing target detection on the video data using a convolutional neural network (CNN), and calculating a first confidence level of the target detection result; Step S107, calculating a first comprehensive environment score at the current moment based on the environment data; Step S109: calculating the data priority at the current moment based on the first confidence level and the first comprehensive environment score; Step S1011: Transmit corresponding data to a ground control station based on the data priority.

[0006] Wherein, the step S105 includes the following steps: Step S1051, calculating the classification probability of targets of different categories; Step S1053, using a smooth L1 function to calculate the loss between the predicted box and the true box; Step S1055: Calculate the first confidence level of the target detection result.

[0007] Wherein, the step S1051 includes: assuming that there is categories, the CNN outputs the classification probability of each category ,in Indicates Category, use the Softmax function to calculate the classification probability of each category: , 1≤j≤N, j is an integer, It is the logits output by the last layer of the CNN.

[0008] Wherein, the step S1053 includes: Use the smooth L1 loss function to calculate the loss between the predicted box and the true box: hour, ,otherwise, , where B pred is the predicted bounding box, is the true bounding box; Calculate the box regression results, .

[0009] Wherein, the step S1055 includes: based on the classification probability and the box regression result, the first confidence calculation formula is: .

[0010] Wherein, the step S107 includes: calculating the first comprehensive environment score by normalization processing : ; Among them, W t , W s and W c is the weight coefficient of temperature, salinity and turbidity in the environmental score, T represents the temperature of the underwater environment, T min Indicates the minimum value of underwater ambient temperature, T max represents the maximum value of the underwater environment temperature, S represents the salinity of the underwater environment, S min Indicates the minimum value of underwater environmental salinity, S maxrepresents the maximum value of underwater salinity, C represents the turbidity of underwater environment, C min Indicates the minimum value of underwater turbidity, C max Indicates the maximum turbidity of the underwater environment.

[0011] Wherein, the step S109 includes calculating the value of the data priority by the following formula: ,in Represents the dynamic weight coefficient of video data, represents the dynamic weight coefficient of sensor data, Indicates the first confidence of the target detection result, Represents the first comprehensive environment score.

[0012] The dynamically adjusted weight coefficients (α_video and α_sensor) are calculated by the first comprehensive environment score and the first confidence of target detection, specifically: ; ; Among them, W target and W environment The weight coefficient for the first confidence and the first comprehensive environment score of target detection.

[0013] Wherein, the step S1011 includes setting the data priority The data is compared with a first threshold, and data corresponding to a data priority greater than the first threshold is transmitted to a ground control station.

[0014] In addition, the present invention also proposes an underwater video robot data acquisition device, comprising A video acquisition module, which is used to obtain video data of the underwater environment in real time; Turbidity sensor, which is used to obtain the turbidity of the underwater environment in real time; A temperature sensor, which is used to obtain the temperature of the underwater environment in real time; A salinity sensor, which is used to obtain the salinity of the underwater environment in real time; A target detection result module, which is used to perform target detection on the video data using a convolutional neural network (CNN) and calculate a first confidence level of the target detection result; An environment score calculation module, which is used to calculate a first comprehensive environment score at a current moment based on the environment data; A priority calculation module, configured to calculate the data priority at a current moment based on the first confidence level and the first comprehensive environment score; A data transmission module is used to transmit corresponding data to a ground control station based on the data priority.

[0015] Compared with the prior art, the present invention has the following advantages: By using convolutional neural networks (CNN), the present invention can identify and classify underwater organisms, obstacles and other environmental features in real time in complex underwater environments. The real-time performance ensures that the robot can respond quickly to environmental changes, improving the efficiency and accuracy of data collection.

[0016] The comprehensive evaluation mechanism combining target detection results and environmental scores can prioritize the transmission of important data and reduce unnecessary data redundancy. This priority evaluation strategy ensures the timely transmission of key data and optimizes the data collection process.

[0017] By dynamically adjusting the weight coefficients (α_video and α_sensor), the present invention can flexibly optimize the data transmission strategy according to real-time environmental changes and target detection results. This adaptability makes the system more stable and reliable in different underwater environments.

[0018] The present invention integrates multiple environmental data such as temperature, salinity and turbidity, and generates an environmental score through normalization processing. This multi-dimensional data analysis capability enables the robot to understand and adapt to the underwater environment more comprehensively, improving the intelligence level of the system.

[0019] By combining sonar data and environmental characteristics, the present invention enables the underwater robot to have the ability of self-navigation and automatic obstacle avoidance. This feature not only improves the safety of the robot, but also enhances its ability to operate in complex underwater environments.

[0020] The dynamic data transmission module adjusts the data transmission rate according to the real-time network status and data priority to ensure the timely transmission of important data. This adaptive mechanism improves the reliability and effectiveness of data transmission and reduces the risk of information loss.

[0021] By uploading data to a cloud computing platform for analysis and storage, and using big data technology to mine historical data, the present invention can provide more powerful support for future underwater data collection and analysis. This capability enables the system to not only be limited to real-time operation, but also to perform long-term data trend analysis.

[0022] The automated data collection and processing process reduces the reliance on manual operations and reduces the errors and risks that may be caused by human intervention. This feature makes underwater robots more reliable in long-term operations and harsh environments.

[0023] Since the underwater organisms and environmental changes can be monitored in real time, the present invention provides an effective technical means for ecological protection and environmental monitoring, which has important practical significance for the research and protection of marine ecosystems. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 The figure is a flow chart showing a method for collecting data of an underwater video robot according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0027] It should be understood that although the terms first, second, third, etc. may be used to describe ... in the embodiments of the present invention, these ... should not be limited to these terms. These terms are only used to distinguish .... For example, without departing from the scope of the embodiments of the present invention, the first ... may also be referred to as the second ..., and similarly, the second ... may also be referred to as the first ....

[0028] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0029] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0030] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the commodity or device including the elements.

[0031] The optional embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0032] Embodiment 1 like Figure 1 As shown, the present invention discloses a method for collecting and evaluating underwater video robot data, comprising the following steps: Step S101, installing a video acquisition device and a plurality of sensors on the underwater video robot; Step S103, real-time acquisition of video data and environmental data, as well as corresponding time information, wherein the environmental data includes turbidity data, temperature and salinity; Step S105: performing target detection on the video data using a convolutional neural network (CNN), and calculating a first confidence level of the target detection result; Step S107, calculating a first comprehensive environment score at the current moment based on the environment data; Step S109: calculating the data priority at the current moment based on the first confidence level and the first comprehensive environment score; Step S1011: Transmit corresponding data to a ground control station based on the data priority.

[0033] Wherein, step S105 includes but is not limited to the following steps.

[0034] Before the video frames are fed into CNN, the images are first preprocessed, including: denoising, using filters to reduce noise in underwater environments; normalization, standardizing the image pixel values ​​to a specific range to increase the convergence speed of the model; and data augmentation, rotating, scaling, flipping, and other operations on the images to increase the diversity of training samples.

[0035] In one embodiment, a convolutional neural network (CNN) is generally composed of multiple layers, including: Convolutional layer, extracts image features and generates feature maps through convolution operations. These features can be low-level features such as edges and textures, which are further combined to form more complex features.

[0036] Activation layer, using nonlinear activation functions (such as ReLU) to enhance the expressiveness of the model.

[0037] The pooling layer reduces the dimension of the feature map, retains important information, and reduces the amount of computation and the risk of overfitting.

[0038] The fully connected layer maps high-dimensional features to classification labels and outputs the final classification results.

[0039] In the present invention, CNN performs target detection through a specific structure (such as YOLO, Faster R-CNN, etc.), which can simultaneously identify multiple targets and generate bounding boxes. Each bounding box corresponds to a detected object and is accompanied by a corresponding classification label.

[0040] CNN generates a confidence score C for each detected object. target , which represents the probability that the target belongs to a specific category. The calculation of confidence is usually based on the following factors: Classification probability, the probability value of each category output by the model; box regression, that is, the accuracy of the bounding box, ensures that the target is correctly located.

[0041] Confidence values ​​usually range from 0 to 1, with higher values ​​indicating that the model is more confident in identifying the target.

[0042] Finally, CNN outputs the recognized target category, bounding box coordinates and its confidence (C_target). The robot uses this information to determine the important objects in the current environment to optimize subsequent data collection and transmission strategies.

[0043] The categories include: underwater life, identifying and classifying fish, corals and other underwater life to provide data support for ecological research; obstacle detection, real-time identification of potential collision risks, such as rocks, sunken ships, etc., to ensure the safe navigation of the robot; environmental feature monitoring: identifying changes in the underwater environment, such as aquatic plants, sediments, etc., to assist in environmental protection and monitoring. The classification can be subdivided or roughly divided into the above three categories, based on demand.

[0044] Embodiment 2 The present invention provides an underwater video robot data collection method, comprising the following steps: Step S101, installing a video acquisition device and a plurality of sensors on the underwater video robot; Step S103, real-time acquisition of video data and environmental data, as well as corresponding time information, wherein the environmental data includes turbidity data, temperature and salinity; Step S105: performing target detection on the video data using a convolutional neural network (CNN), and calculating a first confidence level of the target detection result; Step S107, calculating a first comprehensive environment score at the current moment based on the environment data; Step S109: calculating the data priority at the current moment based on the first confidence level and the first comprehensive environment score; Step S1011: Transmit corresponding data to a ground control station based on the data priority.

[0045] In the present invention, real-time target detection is not only used to identify underwater targets, but also to evaluate the confidence of the target, which can directly affect the priority of the frame video data. Environmental scores (such as water quality, temperature, etc.) will also affect the priority evaluation. By dynamically adjusting the weight (α video and α sensor ), ensuring that important information (such as safety and target detection results) can be processed and transmitted with priority under different environmental conditions.

[0046] In one embodiment, based on real-time video stream, a convolutional neural network (CNN) is used for target detection, identification and tracking of underwater targets (such as marine life, sunken objects, etc.). The loss function used is: , where L represents the loss value, which is used to measure the gap between the model prediction and the true label. Represents the true label, indicating whether the target exists (1 for existence, 0 for non-existence), Represents the model prediction value, and represents the probability of the target existence.

[0047] The convolutional neural network updates the network weights through the following optimization algorithm: , where θ represents the network weight, represents the adjustable parameters in the model, represents the learning rate, which controls the weight update step size, and ∇L(θ) represents the gradient of the loss function, which represents the rate of change of the loss value with respect to the weight.

[0048] Wherein, the step S105 includes the following steps: Step S1051, calculating the classification probability of targets of different categories; Step S1053, using a smooth L1 function to calculate the loss between the predicted box and the true box; Step S1055: Calculate the first confidence level of the target detection result.

[0049] Wherein, the step S1051 includes: assuming that there is categories, the CNN outputs the classification probability of each category ,in Indicates Category, use the Softmax function to calculate the classification probability of each category: , 1≤j≤N, j is an integer, It is the logits output by the last layer of the CNN.

[0050] Wherein, the step S1053 includes: Use the smooth L1 loss function to calculate the loss between the predicted box and the true box: hour, ,otherwise, , where B pred is the predicted bounding box, is the true bounding box; Calculate the box regression results, .

[0051] Wherein, the step S1055 includes: based on the classification probability and the box regression result, the first confidence calculation formula is: .

[0052] Wherein, the step S107 includes: calculating the first comprehensive environment score by normalization processing : ; Among them, W t , W s and W c is the weight coefficient of temperature, salinity and turbidity in the environmental score, T represents the temperature of the underwater environment, T min Indicates the minimum value of underwater ambient temperature, T max represents the maximum value of the underwater environment temperature, S represents the salinity of the underwater environment, S min Indicates the minimum value of underwater environmental salinity, S max represents the maximum value of underwater salinity, C represents the turbidity of underwater environment, C min Indicates the minimum value of underwater turbidity, C max Indicates the maximum turbidity of the underwater environment.

[0053] Wherein, the step S109 includes calculating the value of the data priority by the following formula: ,in Represents the dynamic weight coefficient of video data, represents the dynamic weight coefficient of sensor data, Indicates the first confidence of the target detection result, Represents the first comprehensive environment score.

[0054] The dynamically adjusted weight coefficients (α_video and α_sensor) are calculated by the first comprehensive environment score and the first confidence of target detection, specifically: ; ; Among them, W target and W environment The weight coefficient for the first confidence and the first comprehensive environment score of target detection.

[0055] For example, suppose: Q score = 0.8, T current = 25°C, T min = 15°C, T max = 35°C, S current = 35ppt, S min = 0 ppt, S max = 40 ppt, W q = 0.5, W t = 0.3, W s =0.2; Normalized temperature and salinity: ; Calculate the video data weight: ; Calculate sensor data weights: .

[0056] Wherein, the step S1011 includes setting the data priority The data is compared with a first threshold, and data corresponding to a data priority greater than the first threshold is transmitted to a ground control station.

[0057] In one embodiment, the data of multiple frames are compared, and the D_total values ​​of different time points or frames are compared. The data of the frame or time with a higher D_total value or higher than a certain threshold will be transmitted first, and the transmitted data includes the video data at the current moment or current frame, the environmental data at the current moment or current frame, and the current time information.

[0058] In one embodiment, the environmental score Q score It includes a comprehensive assessment of temperature, salinity and turbidity, where each environmental parameter is normalized using the following formula: , X ∈ {T, S, C}.

[0059] The target detection adopts a multi-layer convolutional neural network structure and optimizes the target recognition capability through incremental learning to improve the detection accuracy and real-time performance in complex underwater environments.

[0060] The dynamic data transmission module adopts an adaptive algorithm to adjust the data transmission rate according to the real-time network status and data priority to ensure the timely transmission of important data.

[0061] The underwater robot has a self-navigation function, and by combining sonar data and environmental data, it can automatically avoid underwater obstacles and optimize the data collection path.

[0062] In one embodiment, the comprehensive evaluation formula further includes a time factor T time , the formula is: , where β is the time weight coefficient, which is used to consider the timeliness of data collection.

[0063] The target detection results are post-processed by an image processing algorithm to eliminate noise and improve the accuracy of target recognition, specifically including morphological processing and edge detection technology.

[0064] The underwater video robot performs data analysis and storage through a cloud computing platform and utilizes big data technology to mine historical data in order to optimize future acquisition strategies and algorithms.

[0065] Embodiment 3 The present invention also proposes an underwater video robot data acquisition device, A video acquisition module, which is used to obtain video data of the underwater environment in real time; Turbidity sensor, which is used to obtain the turbidity of the underwater environment in real time; A temperature sensor, which is used to obtain the temperature of the underwater environment in real time; A salinity sensor, which is used to obtain the salinity of the underwater environment in real time; A target detection result module, which is used to perform target detection on the video data using a convolutional neural network (CNN) and calculate a first confidence level of the target detection result; An environment score calculation module, which is used to calculate a first comprehensive environment score at a current moment based on the environment data; A priority calculation module, configured to calculate the data priority at a current moment based on the first confidence level and the first comprehensive environment score; A data transmission module is used to transmit corresponding data to a ground control station based on the data priority.

[0066] Embodiment 4: An embodiment of the present disclosure provides a non-volatile computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the method steps described in the above embodiment.

[0067] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0068] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0069] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (AN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0070] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0071] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0072] The above introduces the preferred embodiments of the present invention, which is intended to make the spirit of the present invention clearer and easier to understand, but is not intended to limit the present invention. All modifications, substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection outlined by the claims attached to the present invention.

Claims

1. A method for collecting data of an underwater video robot, characterized in that: The following steps are involved: Step S101, installing a video acquisition module and a plurality of sensors on the underwater video robot; Step S103, real-time acquisition of video data and environmental data, as well as corresponding time information, wherein the environmental data includes turbidity data, temperature and salinity; Step S105: performing target detection on the video data using a convolutional neural network (CNN), and calculating a first confidence level of the target detection result; Step S107, calculating a first comprehensive environment score at the current moment based on the environment data; Step S109: calculating the data priority at the current moment based on the first confidence level and the first comprehensive environment score; Step S1011: Transmit corresponding data to a ground control station based on the data priority.

2. The method according to claim 1, characterized in that: The step S105 includes the following steps: Step S1051, calculating the classification probability of targets of different categories; Step S1053, using a smooth L1 function to calculate the loss between the predicted box and the true box; Step S1055: Calculate the first confidence level of the target detection result.

3. The method according to claim 2, characterized in that: The step S1051 includes: assuming there are N categories, the CNN outputs the classification probability of each category ,in Indicates Category, use the Softmax function to calculate the classification probability of each category: , is an integer, It is the logits output by the last layer of the CNN.

4. The method according to claim 3, characterized in that: The step S1053 includes: Use the smooth L1 loss function to calculate the loss between the predicted box and the true box: hour, ,otherwise, , where B pred is the predicted bounding box, is the true bounding box; Calculate the box regression results, .

5. The method according to claim 4, characterized in that: The step S1055 includes: based on the classification probability and the box regression result, the first confidence calculation formula is: .

6. The method according to claim 5, characterized in that The step S107 includes: calculating the first comprehensive environment score by normalization processing : ; Among them, W t , W s and W c is the weight coefficient of temperature, salinity and turbidity in the environmental score, T represents the temperature of the underwater environment, T min Indicates the minimum value of underwater ambient temperature, T max represents the maximum value of the underwater environment temperature, S represents the salinity of the underwater environment, S min Indicates the minimum value of underwater environmental salinity, S max represents the maximum value of underwater salinity, C represents the turbidity of underwater environment, C min Indicates the minimum value of underwater turbidity, C max Indicates the maximum turbidity of the underwater environment.

7. The method according to claim 6, characterized in that The step S109 includes calculating the value of the data priority by the following formula: ,in Represents the dynamic weight coefficient of video data, represents the dynamic weight coefficient of sensor data, Indicates the first confidence of the target detection result, Represents the first comprehensive environment score.

8. The method according to claim 7, characterized in that: The dynamically adjusted weight coefficients (α_video and α_sensor) are calculated by the first comprehensive environment score and the first confidence of target detection, specifically: ; ; Among them, W target and W environment The weight coefficient for the first confidence and the first comprehensive environment score of target detection.

9. The method according to claim 7, characterized in that: The step S1011 includes setting the data priority The data is compared with a first threshold, and data corresponding to a data priority greater than the first threshold is transmitted to a ground control station.

10. An underwater video robot data acquisition device, comprising: A video acquisition device, which is used to acquire video data of an underwater environment in real time; Turbidity sensor, which is used to obtain the turbidity of the underwater environment in real time; A temperature sensor, which is used to obtain the temperature of the underwater environment in real time; A salinity sensor, which is used to obtain the salinity of the underwater environment in real time; A target detection result module, which is used to perform target detection on the video data using a convolutional neural network (CNN) and calculate a first confidence level of the target detection result; An environment score calculation module, which is used to calculate a first comprehensive environment score at a current moment based on the environment data; A priority calculation module, configured to calculate the data priority at a current moment based on the first confidence level and the first comprehensive environment score; A data transmission module is used to transmit corresponding data to a ground control station based on the data priority.

Citation Information

Patent Citations

  • Marine-environment real-time-monitoring system based on underwater robot

    CN108805779A

  • Underwater mobile platform data acquisition and transmission system

    CN109269560A

  • Distributed opportunistic routing method for underwater sensor network based on opto-acoustic fusion

    CN111065144A

  • Road construction environment monitoring method based on Internet of Things

    CN118052666A

  • Information transmission method and device, communication equipment and storage medium

    CN118433819A