Deep learning based underwater vehicle data collection method and apparatus

Abstract

The application discloses a submersible data acquisition method and device based on deep learning, relates to the technical field of underwater detection and artificial intelligence, and internally embeds intelligent sensing and decision control capability in a submersible system, so that the submersible has high autonomy. Even in a deep-sea environment where communication is limited or impossible, the submersible can still independently complete complex target discovery, tracking and fine data acquisition tasks. This not only reduces the dependence on manual operation and remote communication, reduces operation cost and risk, but also greatly expands the operation range and time of the submersible, so that the submersible can undertake longer, longer distance and more complex ocean detection tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of underwater exploration and artificial intelligence, in particular to a submersible data acquisition method and device based on deep learning. BACKGROUND

[0002] During the operation of the submersible, a large amount of image or video data is usually collected for environmental monitoring, target recognition, resource exploration and other tasks. However, due to the complex and changeable underwater environment, poor lighting conditions and serious water scattering, the quality of the collected images is low, and target recognition is difficult. Traditional methods rely on manual image screening and labeling, which is not only inefficient, but also cannot meet the real-time requirements. SUMMARY

[0003] The present application provides a submersible data acquisition method and device based on deep learning, which aims to solve the problem that traditional methods rely on manual image screening and labeling, which is not only inefficient, but also cannot meet the real-time requirements.

[0004] The first aspect of the present application provides a submersible data acquisition method based on deep learning, comprising:

[0005] Obtain historical image data collected by the submersible at a historical time and labels corresponding to the historical image data marked by artificial;

[0006] Construct a deep learning model, and train the deep learning model according to the historical image data and the labels corresponding to the historical image data marked by artificial, to obtain an underwater target recognition model;

[0007] During the operation of the submersible underwater, collect real-time image data with a preset data sampling frequency, and perform image enhancement processing on the real-time image data to obtain real-time image data after enhancement processing;

[0008] Dispatch the underwater target recognition model to recognize the real-time image data after enhancement processing, and determine the target recognition result in the real-time image data;

[0009] Based on the target recognition result in the real-time image data, control the submersible to collect and store data, and complete the submersible data acquisition based on deep learning.

[0010] In a possible design, the deep learning model is constructed, comprising: constructing a YOLO v3 model to obtain a deep learning model.

[0011] In a possible design, the deep learning model is trained according to the historical image data and labels corresponding to the historical image data marked by a person, to obtain an underwater target recognition model, including:

[0012] The hyperparameters of the deep learning model are initialized and coded to obtain a training population;

[0013] For any individual in the training population, the individual is trained according to the historical image data and labels corresponding to the historical image data marked by a person to obtain an individual fitness value;

[0014] Based on the individual fitness value, a first optimal individual in a current training process is determined;

[0015] Based on the first optimal individual, a normal distribution local development strategy is used to locally develop the individual to obtain an individual after local development;

[0016] An information transmission strategy is used to develop the individual after local development by information transmission to obtain an individual after information transmission development;

[0017] A normal distribution global development strategy is used to globally develop the individual after information transmission development to obtain an individual after global development;

[0018] It is determined whether a current training number is greater than or equal to a preset maximum training number, if yes, a second optimal individual is obtained according to the individual after global development, and if not, the step of obtaining the first optimal individual is returned;

[0019] The hyperparameters in the second optimal individual are used as final hyperparameters of the deep learning model to obtain the underwater target recognition model.

[0020] In a possible design, the individual fitness value is obtained according to the historical image data and labels corresponding to the historical image data marked by a person, including:

[0021] The historical image data is used as input data, and the labels corresponding to the historical image data marked by a person are used as expected output to obtain a loss function value;

[0022] The loss function value is added to a preset constant to obtain a non-zero data item, and the non-zero data item is taken as an inverse to obtain the individual fitness value.

[0023] In a possible design, based on the first optimal individual, the normal distribution local development strategy is used to locally develop the individual to obtain the individual after local development, including:

[0024] The average position of the training population in the solution space is obtained as:

[0025]

[0026] wherein, represents the D-dimensional hyperparameter of the i-th individual in the j-th training, k represents the D-dimensional hyperparameter of the i-th individual in the j-th training, i represents the D-dimensional hyperparameter of the i-th individual in the j-th training, d N represents the total number of individuals in the training population, d D represents the total dimension of hyperparameters in the individual, represents the D-dimensional hyperparameter of the average position in the j-th training, k represents the D-dimensional hyperparameter of the average position in the j-th training, d represents the D-dimensional hyperparameter of the average position in the j-th training;

[0027] According to the average position and the first optimal individual, the simulation mean information corresponding to the i-th individual is obtained as: i

[0028]

[0029] wherein, represents the D-dimensional hyperparameter of the simulation mean information corresponding to the i-th individual in the j-th training, k represents the D-dimensional hyperparameter of the simulation mean information corresponding to the i-th individual in the j-th training, i represents the D-dimensional hyperparameter of the simulation mean information corresponding to the i-th individual in the j-th training, d represents the D-dimensional hyperparameter of the first optimal individual; d

[0030] According to the average position, the first optimal individual and the simulation mean information, the simulation standard deviation information corresponding to the i-th individual is obtained as: i

[0031]

[0032] wherein, represents the D-dimensional hyperparameter of the simulation standard deviation information corresponding to the i-th individual, i represents the D-dimensional hyperparameter of the simulation standard deviation information corresponding to the i-th individual, d represents the D-dimensional hyperparameter of the simulation standard deviation information corresponding to the i-th individual,

[0033] The penalty factor is obtained as:

[0034]

[0035] wherein, represents the penalty factor, represents a first random number between 0 and 1, represents a second random number between 0 and 1, represents a third random number between 0 and 1, represents a fourth random number between 0 and 1, represents a cosine function,​​​​ Represents pi (π). Represents a logarithmic function;

[0036] Based on the simulated mean information, simulated standard deviation information, and penalty factor, individuals are locally developed to obtain the following individuals after local development:

[0037]

[0038] in, Indicates the first i The first individual after partial development d Dimensional hyperparameters.

[0039] In one possible design approach, an information delivery strategy is employed to perform information delivery development on the individuals after the partial development, and the individuals after information delivery development are obtained, including:

[0040] The information transmission vector is:

[0041]

[0042]

[0043] in, Indicates the first j The information transmission vector corresponding to the individual after partial development is the first d dimensional hyperparameters, Indicates adaptive inertia weights, This represents the fifth random number that is uniformly distributed between (0,1). Represents the sixth random number uniformly distributed between (0,1). Indicates the first k The first training session i After the individual's partial development, the first individual's d dimensional hyperparameters, Indicates the first j The Euclidean distance between an individual after partial development and the first optimal individual; Indicates the maximum number of training iterations. This represents a random number that follows a standard normal distribution and is distributed according to the first normal distribution. This represents the seventh random number between (0,1). This represents the eighth random number between (0, 1); Represents the first individual after random local development. d dimensional hyperparameters;

[0044] Based on the information transmission vector, the information transmission individual is obtained as follows:

[0045]

[0046] wherein, denotes the j dimensional hyper-parameters of the individual after the d local development corresponding to the information passing individual;

[0047] According to the information passing individual, the individual after the local development is developed by information passing development, and the individual after the information passing development is obtained as:

[0048]

[0049] wherein, denotes the j individual after the information passing development, j denotes the fitness of the individual after the local development, j denotes the fitness of the information passing individual corresponding to the individual after the local development, j denotes the Euclidean distance between the individual after the

[0050] In a possible design, the individual after the information passing development is developed globally by adopting a normal distribution global development strategy, and the individual after the global development is obtained, comprising:

[0051] The individual after the information passing development is randomly matched with three other individuals after the information passing development, so as to obtain a first tracking individual, a second tracking individual and a third tracking individual corresponding to each individual after the information passing development;

[0052] According to the first tracking individual, the second tracking individual and the third tracking individual, a first tracking vector and a second tracking vector are obtained as:

[0053]

[0054]

[0055] wherein, denotes the k dimensional hyper-parameters of the individual after the m information passing development at the d time of training, denotes the d dimensional hyper-parameters of the first tracking individual, denotes the d dimensional hyper-parameters of the first tracking vector, denotes the k dimensional hyper-parameters of the second tracking vector, andm The fitness of an individual after the development of information transmission. Indicates the fitness of the first tracked individual; The second tracking vector represents the first... d dimensional hyperparameters, The second tracking vector represents the first... d dimensional hyperparameters, The third tracking vector represents the... d dimensional hyperparameters, This indicates the fitness of the second tracked individual. This indicates the fitness of the third tracked individual;

[0056] Based on the first tracking vector and the second tracking vector, global development is performed on the individual after the information transmission development, and the individual after global development is obtained as follows:

[0057]

[0058] in, Indicates the first m The first individual after global development d dimensional hyperparameters, Represents the tenth random number between (0,1). This represents a random number that follows a standard normal distribution or a second normal distribution. This represents a random number that follows a standard normal distribution but is distributed in the third normal distribution.

[0059] In one possible design approach, the real-time image data is subjected to image enhancement processing to obtain enhanced real-time image data, including:

[0060] The real-time image data is subjected to three-channel color compensation, global histogram stretching, and / or HSV space enhancement to obtain enhanced real-time image data.

[0061] In one possible design approach, based on the target recognition results in the real-time image data, the submersible is controlled to perform data acquisition and storage, including:

[0062] If the target recognition result in the real-time image data contains a preset target, the data sampling frequency is increased, and the increased data sampling frequency is used to collect real-time image data, or the recording function is enabled to collect real-time video data, and the collected real-time image data and real-time video data are stored.

[0063] A second aspect of this application provides a deep learning-based submersible data acquisition device, comprising:

[0064] The training data collection module is configured to collect historical image data collected by the submersible at a historical time and labels corresponding to the historical image data marked by a human being.

[0065] The deep learning model is configured to construct a deep learning model and train the deep learning model according to the historical image data and the labels corresponding to the historical image data marked by the human being, to obtain an underwater target recognition model.

[0066] The image enhancement module is configured to collect real-time image data at a preset data sampling frequency during the operation of the submersible under water, and perform image enhancement processing on the real-time image data to obtain real-time image data after enhancement processing.

[0067] The target recognition module is configured to schedule the underwater target recognition model to recognize the real-time image data after enhancement processing, to determine a target recognition result in the real-time image data.

[0068] The submersible data collection module is configured to control the submersible to collect and store data based on the target recognition result in the real-time image data, to complete submersible data collection based on deep learning.

[0069] Beneficial effects:

[0070] The application provides a submersible data collection method and device based on deep learning, which embeds intelligent perception and decision control capability in a submersible system, so that the submersible has high autonomy. Even in a deep sea environment where communication is limited or impossible, the submersible can still independently complete complex target discovery, tracking and fine data collection tasks. This not only reduces the dependence on manual operation and remote communication, reduces operation cost and risk, but also greatly expands the operation range and duration of the submersible, so that it can undertake longer, longer distance and more complex ocean exploration tasks. BRIEF DESCRIPTION OF DRAWINGS

[0071] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the application. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0072] Figure 1 is a flowchart of a submersible data collection method based on deep learning according to an embodiment of the application;

[0073] Figure 2is a structural schematic diagram of a submersible data acquisition device based on deep learning, which is proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0075] As shown in the drawings, Figure 1 The present application provides a submersible data acquisition method based on deep learning, which comprises the following steps.

[0076] S101, obtaining historical image data collected by a submersible at a historical time and labels corresponding to the historical image data marked by a human being; wherein the historical image data is data after image enhancement processing.

[0077] A large amount of historical image data collected by a submersible in previous tasks is obtained, and these images are manually marked by marine biologists, geologists or professional annotators. The marking content is the target of interest in the image, such as "coral", "fish A", "wreck", "hydrothermal vent", etc., and the position (bounding box) is marked. In particular, after obtaining these original historical image data, the original historical image data needs to be processed by image enhancement processing, and then the enhanced image and the corresponding label are used as the final training data. This is done to eliminate environmental interference in the original image, unify the data distribution, and improve the effect of subsequent model training. It is worth noting that how to annotate the training data of the YOLO model is a relatively conventional technology, and the embodiments of the present application will not be described in detail.

[0078] S102, constructing a deep learning model, and training the deep learning model according to the historical image data and the labels corresponding to the historical image data marked by a human being to obtain an underwater target recognition model.

[0079] The embodiments of the present application can select a YOLO v3 model with speed and accuracy in the target detection field as a basic deep learning model. After the model framework is constructed, the model is trained using the enhanced historical image data and labels prepared in step S101. The core of the training process is to optimize the hyperparameters of the model to obtain an underwater target recognition model with the best performance.

[0080] S103, during the running process of the submersible underwater, real-time image data is collected at a preset data sampling frequency, and image enhancement processing is performed on the real-time image data to obtain real-time image data after image enhancement processing.

[0081] When the submersible performs an underwater exploration task, the camera carried by the submersible continuously collects real-time image data at a preset, lower basic data sampling frequency (for example, taking a photo every 2 seconds). Due to the optical properties of the underwater environment, the quality of these original real-time images is usually poor. Therefore, the image processing unit of the submersible immediately performs image enhancement processing on each frame of real-time image.

[0082] S104, scheduling the underwater target recognition model to recognize the real-time image data after the enhancement processing, and determining a target recognition result in the real-time image data.

[0083] The real-time image data after the enhancement processing in step S103 is input into the underwater target recognition model that has been trained in step S102. The model performs inference operation on the computing platform of the submersible, and quickly outputs the target recognition result of the current image. The recognition result usually includes the detected target category and the position (bounding box coordinates) of the target in the image.

[0084] S105, based on the target recognition result in the real-time image data, controlling the submersible to perform data collection and storage, and completing the submersible data collection based on deep learning.

[0085] The present application provides a submersible data collection method based on deep learning, which embeds intelligent perception and decision control capability in the submersible system, so that the submersible has high autonomy. Even in a deep-sea environment where communication is limited or impossible, the submersible can still independently complete complex target discovery, tracking and fine data collection tasks. This not only reduces the dependence on manual operation and remote communication, reduces operation cost and risk, but also greatly expands the operation range and duration of the submersible, enabling it to undertake longer, longer distance and more complex ocean exploration tasks.

[0086] In a possible design, the deep learning model is constructed, including: constructing a YOLO v3 model to obtain the deep learning model. It should be noted that the YOLO v3 model is only a preferred embodiment of the present application, and other models can also be used to construct the deep learning model.

[0087] In a possible design, the deep learning model is trained according to the historical image data and the labels corresponding to the historical image data marked by humans to obtain an underwater target recognition model, including:

[0088] The hyperparameters of the deep learning model are initialized and coded to obtain a training population;

[0089] The method for obtaining the training population can refer to the prior art, and details are not described herein.

[0090] For any individual in the training population, the fitness corresponding to the individual is obtained according to the historical image data and the label corresponding to the historical image data marked by a person.

[0091] Based on the fitness corresponding to the individual, a first optimal individual in the current training process is determined.

[0092] Based on the first optimal individual, a local development strategy of normal distribution is used to develop the individual locally, and an individual after local development is obtained.

[0093] An information transmission strategy is used to develop the individual after local development, and an individual after information transmission development is obtained.

[0094] A global development strategy of normal distribution is used to develop the individual after information transmission development, and an individual after global development is obtained.

[0095] It is judged whether the current training number is greater than or equal to a preset maximum training number, if yes, a second optimal individual is obtained according to the individual after global development, otherwise the step of obtaining the first optimal individual is returned.

[0096] The hyperparameters in the second optimal individual are taken as the final hyperparameters of the deep learning model, and an underwater target recognition model is obtained.

[0097] In a possible design, the fitness corresponding to the individual is obtained according to the historical image data and the label corresponding to the historical image data marked by a person, and includes:

[0098] The historical image data is taken as input data, and the label corresponding to the historical image data marked by a person is taken as expected output, and a loss function value is obtained.

[0099] The loss function value is added to a preset constant (such as 0.001 or 0.0001) to obtain a non-zero data item, and the non-zero data item is taken as an inverse to obtain the fitness corresponding to the individual.

[0100] In a possible design, based on the first optimal individual, the local development strategy of normal distribution is used to develop the individual locally, and the individual after local development is obtained, and includes:

[0101] The average position of the training population in the solution space is:

[0102]

[0103] wherein, represents the first k dimensional hyperparameter of the i-th individual in the j-th training, i represents the first d dimensional hyperparameter of the i-th individual in the j-th training, d N represents the total number of individuals in the training population, represents the first k dimensional hyperparameter of the average position in the j-th training, d represents the first dimensional hyperparameter of the average position in the j-th training,

[0104] i According to the average position and the first optimal individual, the simulation mean information corresponding to the i-th individual is obtained as:

[0105]

[0106] wherein, represents the first k dimensional hyperparameter of the simulation mean information corresponding to the i-th individual in the j-th training, i represents the first d dimensional hyperparameter of the simulation mean information corresponding to the i-th individual in the j-th training, represents the first d dimensional hyperparameter of the first optimal individual;

[0107] According to the average position, the first optimal individual and the simulation mean information, the simulation standard deviation information corresponding to the i-th individual is obtained as: i

[0108]

[0109] wherein, represents the first i dimensional hyperparameter of the simulation standard deviation information corresponding to the i-th individual; d The penalty factor is obtained as:

[0110]

[0111]

[0112] wherein, represents the penalty factor, represents a first random number between (0, 1), represents a second random number between (0, 1), represents a third random number between (0, 1), represents a fourth random number between (0, 1), represents a cosine function, represents a constant pi, represents a logarithmic function;

[0113] ​​According to the simulation mean information, simulation standard deviation information and the penalty factor, the individual is locally developed, and the individual after local development is obtained as:

[0114]

[0115] wherein, represents the i-th dimension hyperparameter of the i-th individual after local development. i d

[0116] By introducing the population average position, the optimal individual information and the dynamically changing penalty factor, the individual is guided to perform fine normal distribution search near the current optimal solution. This strategy can quickly converge to the local optimal region and deeply mine, thereby improving the optimization precision and efficiency.

[0117] In a possible design manner, the information transmission strategy is adopted to develop the individual after local development, and the individual after information transmission development is obtained, including:

[0118] The information transmission vector is obtained as:

[0119]

[0120]

[0121] wherein, represents the i-th dimension hyperparameter of the i-th individual after local development. j d represents the i-th dimension hyperparameter of the i-th individual after local development. represents the adaptive inertia weight, represents the fifth random number uniformly distributed between 0 and 1, represents the sixth random number uniformly distributed between 0 and 1, represents the i-th dimension hyperparameter of the i-th individual after local development in the t-th training, k i represents the i-th dimension hyperparameter of the i-th individual after local development in the t-th training, d represents the Euclidean distance between the i-th individual after local development and the first optimal individual; represents the maximum training number, j represents the first normal distribution random number subject to standard normal distribution, represents the seventh random number between 0 and 1, represents the eighth random number between 0 and 1; represents the i-th dimension hyperparameter of the random individual after local development; d

[0122] ​​​​​​​According to the information transmission vector, an information transmission individual is obtained as:

[0123]

[0124] wherein, denotes the fitness of the individual after the kth local development, j denotes the kth dimension hyperparameter of the individual corresponding to the information transmission individual after the kth local development, d

[0125] According to the information transmission individual, the individual after the local development is developed by information transmission, and an individual after information transmission development is obtained as:

[0126]

[0127] wherein, denotes the individual after the kth information transmission development, j denotes the fitness of the individual after the kth local development, denotes the fitness of the individual corresponding to the information transmission individual after the kth local development, j denotes the Euclidean distance between the individual after the kth local development and the information transmission individual thereof. j The individual not only learns from the current optimal individual, but also performs information transmission with a random individual, and is adjusted by an adaptive inertia weight and a distance factor, which effectively avoids the rapid loss of population diversity, prevents the algorithm from falling into "premature" and local optimum, and enhances the ability of the algorithm to jump out of the local optimal solution. j In a possible design, a normal distribution global development strategy is adopted to develop the individual after the information transmission development, and an individual after global development is obtained, including:

[0128] The individual after the information transmission development is randomly matched with three other individuals after the information transmission development, and the first tracking individual, the second tracking individual and the third tracking individual corresponding to each individual after the information transmission development are obtained;

[0129] According to the first tracking individual, the second tracking individual and the third tracking individual, a first tracking vector and a second tracking vector are obtained as:

[0130]

[0131]

[0132]

[0133]

[0134] wherein, denotes the kth dimension hyperparameter of the individual after the kth local development,​​​​​k the first tracking individual, m the first tracking individual, d a d-th dimension hyper-parameter of the first tracking individual, a d-th dimension hyper-parameter of the first tracking individual, d a d-th dimension hyper-parameter of the first tracking individual, a d-th dimension hyper-parameter of the first tracking individual, d a d-th dimension hyper-parameter of the first tracking individual, a d-th dimension hyper-parameter of the first tracking individual, k a d-th dimension hyper-parameter of the first tracking individual, m a d-th dimension hyper-parameter of the first tracking individual, a d-th dimension hyper-parameter of the first tracking individual, a d-th dimension hyper-parameter of the first tracking individual, d a d-th dimension hyper-parameter of the first tracking individual, a d-th dimension hyper-parameter of the first tracking individual, d a d-th dimension hyper-parameter of the first tracking individual, a d-th dimension hyper-parameter of the first tracking individual, d a d-th dimension hyper-parameter of the first tracking individual, a d-th dimension hyper-parameter of the first tracking individual, a d-th dimension hyper-parameter of the first tracking individual,

[0135] According to the first tracking vector and the second tracking vector, the individual after the information transmission development is globally developed, and the individual after the global development is obtained as:

[0136]

[0137] wherein, a d-th dimension hyper-parameter of the first tracking individual, m a d-th dimension hyper-parameter of the first tracking individual, d a d-th dimension hyper-parameter of the first tracking individual, a tenth random number between (0, 1), a second normal distribution random number subject to a standard normal distribution, a third normal distribution random number subject to a standard normal distribution.

[0138] Optionally, a greedy strategy can also be used to control the global development process, and after each strategy is executed, the individual can be processed to ensure that the hyper-parameter is always within its upper and lower limits.

[0139] By randomly selecting multiple individuals and constructing tracking vectors based on the fitness difference, the individuals are guided to explore a large range in the solution space. The introduction of normal distribution increases the randomness and breadth of the search, ensuring that the algorithm can effectively explore the global range and thus has a greater probability of finding the global optimal solution.

[0140] Through the organic combination and iterative execution of the three strategies, the embodiments of the application can automatically and efficiently find a set of optimal hyperparameter combinations for the YOLOv3 model, so that the trained underwater target recognition model is superior to the model trained by the traditional method in terms of key indicators such as recognition accuracy, recall rate and average precision, maximizes the performance of the model, and ensures the accuracy and integrity of data collection, which can effectively avoid data missing collection.

[0141] In a possible design, the real-time image data is subjected to image enhancement processing to obtain real-time image data after enhancement processing, including:

[0142] The real-time image data is subjected to three-channel color compensation, global histogram stretching processing and / or HSV space enhancement processing to obtain real-time image data after enhancement processing.

[0143] Three-channel color compensation: According to the absorption and attenuation law of red, green and blue light of water at different depths, different degrees of gain compensation are performed on the RGB three channels of the image, especially the red light channel which is severely absorbed, to restore the true color of the object. For example, the red and blue channels of the underwater image can be subjected to equalization processing.

[0144] Global histogram stretching: The global histogram of the image is calculated and stretched to the entire dynamic range to increase the overall contrast of the image, so that the dark details are clearer and the bright parts are not overexposed. For example, the processed three-channel values can be linearly mapped to the range of 0-1 to complete the overall normalization processing, thereby improving the visibility of the low-illumination area in the image and compensating for the color attenuation and improving the contrast.

[0145] HSV space enhancement processing: The image is converted from RGB space to HSV (hue, saturation, lightness) space. The saturation channel is moderately improved to make the colors more vivid, and the lightness channel is subjected to gamma correction or histogram equalization to improve the problem of uneven illumination.

[0146] It should be noted that the above three image enhancement methods are only preferred modes of the embodiments of the application, and other image enhancement methods can also be used to process the image data to improve the data recognition accuracy.

[0147] The embodiments of the present application introduce image enhancement processing in both model training and real-time application stages. Before training, the historical image data is enhanced, which can effectively eliminate the color and brightness differences of images in different batches and different environments, construct a more unified and high-quality data set, and provide protection for the model to learn essential features rather than environmental noise. When collecting real-time data, the same three-channel color compensation, global histogram stretching, and HSV space enhancement processing are performed on the original image, which can effectively correct the color distortion of underwater images, enhance the contrast and details, and greatly improve the quality of images input to the recognition model. This "end-to-end" image enhancement strategy fundamentally overcomes the interference of water optical properties on visual recognition, enabling the YOLO v3 model to maintain high-precision and high-stability target recognition capability in complex and variable underwater environments.

[0148] In a possible design, based on the target recognition result in the real-time image data, the submersible is controlled to collect and store data, including:

[0149] If the target recognition result in the real-time image data contains a preset target, the data sampling frequency is increased, the real-time image data is collected by using the increased data sampling frequency, or the video recording function is started to collect real-time video data, and the collected real-time image data and real-time video data are stored.

[0150] The embodiments of the present application establish an intelligent closed loop of "recognition-decision-action". The submersible is no longer blindly executing a preset program, but uses the high-performance underwater target recognition model carried by the submersible to perform real-time perception and understanding of the environment. When a preset target of interest (such as rare animals, archaeological relics, minerals, etc.) is recognized, the system can immediately make an intelligent decision.

[0151] Dynamic adjustment of sampling strategy: automatically increase the image sampling frequency, switch from "slow scanning" to "high-speed staring", and capture more transient and high-resolution detail information of the target.

[0152] Trigger advanced collection mode: automatically start the video recording function, upgrade discrete image data to continuous video stream, and completely record the dynamic behavior of the target or the interaction process with the environment, which has irreplaceable value for biological research and target dynamic analysis.

[0153] This on-demand collection and intelligent triggering working mode completely changes the traditional data collection method of the submersible, reduces the redundant data collection in non-value areas, saves up to 70% of storage space and energy consumption; at the same time, it ensures the comprehensive and high-quality capture of key target data, significantly improves the scientific value and commercial value density of the collected data.

[0154] AsFigure 2 As shown, based on the same inventive concept, the embodiment of the present application also provides a submersible data acquisition device based on deep learning, comprising:

[0155] The training data acquisition module 201 is configured to acquire historical image data collected by the submersible at a historical time and labels corresponding to the historical image data marked by an artificial person; wherein the historical image data is data after image enhancement processing.

[0156] The deep learning model 202 is configured to construct a deep learning model, train the deep learning model according to the historical image data and the labels corresponding to the historical image data marked by the artificial person, and obtain an underwater target recognition model.

[0157] The image enhancement module 203 is configured to collect real-time image data by using a preset data sampling frequency during the operation of the submersible under water, and perform image enhancement processing on the real-time image data to obtain real-time image data after enhancement processing.

[0158] The target recognition module 204 is configured to schedule the underwater target recognition model to recognize the real-time image data after enhancement processing, and determine a target recognition result in the real-time image data.

[0159] The submersible data acquisition module 205 is configured to control the submersible to collect and store data based on the target recognition result in the real-time image data, and complete the submersible data acquisition based on deep learning.

[0160] The submersible data acquisition device based on deep learning provided by the embodiment of the present application can execute the above method technical solution, and has similar principles and beneficial effects, which will not be repeated here.

[0161] The embodiment of the present application also provides a computer readable storage medium, which stores computer programs / instructions, and the computer programs / instructions are executed by a processor to realize the steps in the method disclosed in the embodiment of the present application.

[0162] The embodiment of the present application also provides a computer program product, which, when running on an electronic device, causes a processor to execute the steps in the method disclosed in the embodiment of the present application.

[0163] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts between each embodiment can be referred to each other.

[0164] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic device and computer program products according to the embodiments of the present application. It is understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowcharts and / or block diagrams block or blocks. Figure 1 one or more flows and / or blocks Figure 1 an apparatus with a function specified in one or more flows and / or blocks.

[0165] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal devices to work in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction apparatus that implements the functions specified in the flowcharts and / or block diagrams block or blocks. Figure 1 one or more flows and / or blocks Figure 1 an apparatus with a function specified in one or more flows and / or blocks.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are performed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in the flowcharts and / or block diagrams block or blocks. Figure 1 one or more flows and / or blocks Figure 1 an apparatus with a function specified in one or more flows and / or blocks.

[0167] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0168] Finally, it needs to be pointed out that in this document, relational terms such as first and second and the like can only be intended to distinguish one entity or operation from another entity or operation without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprising", "comprising" or any other variations thereof are intended to cover non-exclusive inclusions, so that a process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or terminal device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the element.

[0169] The principles and implementation manners of the present application are described by applying specific examples in this document, and the above example descriptions are only used to help understand the method and its core idea of the present application; meanwhile, for the general technical personnel in the field, the specific implementation manners and application ranges will be changed according to the idea of the present application, and the above description should not be understood as the limitation of the present application.

Claims

1. A deep learning-based submersible data acquisition method, characterized in that, include: Acquire historical image data collected by the submersible at historical moments and manually labeled tags corresponding to the historical image data; wherein, the historical image data is data after image enhancement processing; A deep learning model is constructed and trained based on the historical image data and manually labeled information corresponding to the historical image data to obtain an underwater target recognition model. During the underwater operation of the submersible, real-time image data is acquired using a preset data sampling frequency, and the real-time image data is subjected to image enhancement processing to obtain enhanced real-time image data. The underwater target recognition model is scheduled to recognize the enhanced real-time image data to determine the target recognition result in the real-time image data; Based on the target recognition results in the real-time image data, the submersible is controlled to collect and store data, thus completing the deep learning-based submersible data collection. The deep learning model is trained based on the historical image data and manually labeled tags corresponding to the historical image data to obtain an underwater target recognition model, including: The hyperparameters of the deep learning model are initialized and encoded to obtain the training population; For any individual in the training population, the fitness of the individual is obtained by training based on the historical image data and manually labeled tags corresponding to the historical image data. Based on the fitness of the individual, determine the first optimal individual in the current training process; Based on the first optimal individual, a normal distribution local development strategy is used to locally develop the individual to obtain the locally developed individual; An information transmission strategy is used to perform information transmission development on the individuals after the partial development, and to obtain the individuals after the information transmission development; A normal distribution global development strategy is adopted to perform global development on the individuals after the information transmission development, and the individuals after global development are obtained. Determine if the current number of training iterations is greater than or equal to the preset maximum number of training iterations. If so, obtain the second optimal individual based on the individuals after global development; otherwise, return to the step of obtaining the first optimal individual. The hyperparameters in the second optimal individual are used as the final hyperparameters of the deep learning model to obtain the underwater target recognition model; Based on the first optimal individual, a normal distribution local development strategy is used to locally develop the individual, obtaining the locally developed individual, including: The average position of the training population in the solution space is obtained as follows: in, Indicates the first k The first training session i The first individual d The hyperparameter is N, which represents the total number of individuals in the training population. d =1,2,…,D, where D represents the total dimension of hyperparameters in an individual. Indicates the first k The average position during the training cycle. d dimensional hyperparameters; Based on the average position and the first optimal individual, obtain the first... i The simulated mean information for each individual is: in, Indicates the first k During the training session i The simulated average information corresponding to each individual d dimensional hyperparameters, The first optimal individual represents the first optimal individual. d dimensional hyperparameters; Based on the average position, the first optimal individual, and the simulated mean information, obtain the first... i The simulated standard deviation information for each individual is as follows: in, Indicates the first i The simulated standard deviation information for each individual. d dimensional hyperparameters; The penalty factor is: in, Indicates the penalty factor. Represents the first random number between (0,1). This represents the second random number between (0,1). This represents a third random number between (0,1). This represents the fourth random number between (0,1). Represents the cosine function. Represents pi (π). Represents a logarithmic function; Based on the simulated mean information, simulated standard deviation information, and penalty factor, individuals are locally developed to obtain the following individuals after local development: in, Indicates the first i The first individual after partial development d Dimensional hyperparameters.

2. The deep learning-based submersible data acquisition method according to claim 1, characterized in that, Building a deep learning model includes: building the YOLO v3 model to obtain the deep learning model.

3. The deep learning-based submersible data acquisition method according to claim 1, characterized in that, Based on the historical image data and manually labeled tags corresponding to the historical image data, the fitness of an individual is obtained, including: Using the historical image data as input data, and training with manually labeled tags corresponding to the historical image data as the expected output, the loss function value is obtained; The loss function value is added to a preset constant to obtain a non-zero data item, and the reciprocal of the non-zero data item is taken to obtain the fitness of the individual.

4. The deep learning-based submersible data acquisition method according to claim 1, characterized in that, An information delivery strategy is employed to perform information delivery development on the individuals after the partial development, and to obtain the individuals after the information delivery development, including: The information transmission vector is: in, Indicates the first j The information transmission vector corresponding to the individual after partial development is the first d dimensional hyperparameters, Indicates adaptive inertia weights, This represents the fifth random number that is uniformly distributed between (0,1). Represents the sixth random number uniformly distributed between (0,1). Indicates the first k The first training session i After the individual's partial development, the first individual's d dimensional hyperparameters, Indicates the first j The Euclidean distance between an individual after partial development and the first optimal individual; Indicates the maximum number of training iterations. This represents a random number that follows a standard normal distribution and is distributed according to the first normal distribution. This represents the seventh random number between (0,1). This represents the eighth random number between (0, 1); Represents the first individual after random local development. d dimensional hyperparameters; Based on the information transmission vector, the information transmission individual is obtained as follows: in, Indicates the first j The information transmission of the individual corresponding to the individual after partial development. d dimensional hyperparameters; Based on the information transmission individual, information transmission development is performed on the individual after the partial development, and the individual after information transmission development is obtained as follows: in, Indicates the first j An individual after information transmission development, Indicates the first j The fitness of an individual after partial development. Indicates the first j The fitness of an individual after partial development, corresponding to information transmission. Indicates the first j The Euclidean distance between an individual after partial development and the individual transmitting information.

5. The deep learning-based submersible data acquisition method according to claim 4, characterized in that, A normal distribution global development strategy is used to perform global development on the individuals after the information transmission development, and the individuals after global development are obtained, including: For each individual that has undergone information transmission development, three other individuals that have undergone information transmission development are randomly matched to obtain the first tracking individual, the second tracking individual, and the third tracking individual corresponding to each individual that has undergone information transmission development. Based on the first tracking individual, the second tracking individual, and the third tracking individual, the first tracking vector and the second tracking vector are obtained as follows: in, Indicates the first k During the training session m The first individual after information transmission development d dimensional hyperparameters, This indicates the first individual being tracked. d dimensional hyperparameters, Represents the first tracking vector's... d dimensional hyperparameters, Indicates the first k During the training session m The fitness of an individual after the development of information transmission. Indicates the fitness of the first tracked individual; The second tracking vector represents the first... d dimensional hyperparameters, The second tracking vector represents the first... d dimensional hyperparameters, The third tracking vector represents the... d dimensional hyperparameters, This indicates the fitness of the second tracked individual. This indicates the fitness of the third tracked individual; Based on the first tracking vector and the second tracking vector, global development is performed on the individual after the information transmission development, and the individual after global development is obtained as follows: in, Indicates the first m The first individual after global development d dimensional hyperparameters, Represents the tenth random number between (0,1). This represents a random number that follows a standard normal distribution or a second normal distribution. This represents a random number that follows a standard normal distribution but is distributed in the third normal distribution.

6. The deep learning-based submersible data acquisition method according to claim 1, characterized in that, The real-time image data is subjected to image enhancement processing to obtain enhanced real-time image data, including: The real-time image data is subjected to three-channel color compensation, global histogram stretching, and / or HSV space enhancement to obtain enhanced real-time image data.

7. The deep learning-based submersible data acquisition method according to claim 1, characterized in that, Based on the target recognition results in the real-time image data, the submersible is controlled to perform data acquisition and storage, including: If the target recognition result in the real-time image data contains a preset target, the data sampling frequency is increased, and the increased data sampling frequency is used to collect real-time image data, or the recording function is enabled to collect real-time video data, and the collected real-time image data and real-time video data are stored.

8. A deep learning-based submersible data acquisition device, wherein the deep learning-based submersible data acquisition device is capable of executing the deep learning-based submersible data acquisition method according to any one of claims 1 to 7, characterized in that, include: The training data acquisition module is used to acquire historical image data collected by the submersible at historical moments and manually labeled tags corresponding to the historical image data; wherein, the historical image data is data after image enhancement processing; A deep learning model is used to construct a deep learning model and train the deep learning model based on the historical image data and manually labeled tags corresponding to the historical image data to obtain an underwater target recognition model. The image enhancement module is used to acquire real-time image data at a preset data sampling frequency during the underwater operation of the submersible, and to perform image enhancement processing on the real-time image data to obtain enhanced real-time image data. The target recognition module is used to schedule the underwater target recognition model to recognize the enhanced real-time image data and determine the target recognition result in the real-time image data. The submersible data acquisition module is used to control the submersible to acquire and store data based on the target recognition results in the real-time image data, thereby completing the deep learning-based submersible data acquisition.

Images