Marine ecological environment detection method, device and equipment and storage medium

Abstract

The invention discloses a marine ecological environment detection method and device, equipment and a storage medium, and relates to the technical field of artificial intelligence and marine ecological monitoring, in the method, a target task scene is determined, scene description information of the target task scene is obtained, various marine environment data collected in the target task scene is obtained, and the scene description information is stored in the target task scene; based on the scene description information and the attribute information of the various types of marine environment data, obtaining a comprehensive quality evaluation value of the actual marine environment, selecting a data fusion mode matched with the comprehensive quality evaluation value, and based on the data fusion mode, performing fusion processing on the various types of marine environment data to obtain multi-modal fusion environment data. And analyzing the fusion environment data to obtain a detection result of the target task scene. Through the mode, the multi-modal marine environment data can be fused for the target task scene, the data collaboration advantage is fully played, the limitation of single data is avoided, and the precision and adaptability of marine ecological environment detection are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and marine ecological monitoring technology, in particular to a marine ecological environment detection method, device, equipment and storage medium. BACKGROUND

[0002] With the deepening of marine scientific research and the acceleration of marine resource development, the importance of marine ecological monitoring is increasingly prominent, and the current marine ecological monitoring system is developing towards intelligence and automation.

[0003] In related technologies, in order to cope with the problem of limited detection ability and adaptation range of a single sensor, marine ecological monitoring has gradually introduced multi-sensor cooperation for marine ecological monitoring. By integrating multiple sensors (such as sonar, visible light camera and laser radar) in the underwater sensing terminal, multi-modal data (such as acoustic data, optical image data and three-dimensional point cloud data) of underwater targets are collected to obtain multi-dimensional information of targets under different water conditions, thereby making up for the limitations of single sensors in detection distance, imaging accuracy and environmental adaptability.

[0004] In the above scheme, each sensor collects corresponding modal data according to preset parameters, and then stores the collected multi-modal data simply. Subsequently, according to specific scene requirements (such as calling acoustic data in deep sea environment, calling optical image data in clear water body, and calling laser radar data in shallow sea and medium depth water body), a certain modal data is selected, and then a monitoring task is performed relying on the modal data.

[0005] However, since the above scheme independently calls each modal data, the advantages of relying on single modal data are adapted to the corresponding scene (such as long distance detection of acoustic data, detail presentation of optical image data, or high precision shape capture of laser radar data). Therefore, in the process of marine ecological monitoring, once facing dynamic changes of environment or diversity monitoring tasks, the advantages of each modal data cannot form synergistic complementation.

[0006] For example, when monitoring the fish population in shallow sea, the initial clear water body calls optical image data to identify fish species. If the sudden increase of plankton causes the water body to be turbid, the optical image data is invalid, and acoustic data is called instead. In the whole monitoring process, the system collects multi-modal data such as optical image and acoustic data, but in actual application, only one modal data is called according to the scene, and only single modal information can be obtained, thereby affecting the comprehensiveness of monitoring and the accuracy of analysis, and reducing the task adaptation effect.

[0007] Therefore, it is necessary to provide a new marine ecological environment detection method to overcome the above defects. SUMMARY

[0008] The application provides a marine ecological environment detection method, device, equipment and storage medium, which can avoid independent calling of multi-modal data, inability of cooperation and complementation, and problems of insufficient comprehensive detection and poor task adaptation effect.

[0009] In a first aspect, the application provides a marine ecological environment detection method, which comprises: determining a target task scene and obtaining scene description information of the target task scene; the scene description information is used to describe an actual marine environment under the target task scene; obtaining various types of marine environment data collected under the target task scene; different types of marine environment data have different perception modalities; based on the scene description information and attribute information of the various types of marine environment data, obtaining a comprehensive quality evaluation value of the actual marine environment, and selecting a data fusion mode matched with the comprehensive quality evaluation value; based on the data fusion mode, performing fusion processing on the various types of marine environment data to obtain fusion environment data with multiple modalities; analyzing the fusion environment data to obtain a detection result of the target task scene.

[0010] In an optional embodiment, the method further comprises: after obtaining the various types of marine environment data collected under the target task scene, before obtaining the comprehensive quality evaluation value of the actual marine environment based on the scene description information and the attribute information of the various types of marine environment data, the method further comprises: for each type of marine environment data, respectively performing: from the marine environment data, deleting part of the data that does not meet a preset correlation degree condition with the target task scene to obtain updated marine environment data.

[0011] In an optional embodiment, the method further comprises: the target task scene is to detect hydrological parameters of a marine environment, and the actual marine environment described by the scene description information is a hydrological comprehensive interference condition; based on the scene description information and the attribute information of the various types of marine environment data, obtaining the comprehensive quality evaluation value of the actual marine environment, and selecting a data fusion mode matched with the comprehensive quality evaluation value, comprises: for each type of marine environment data, respectively performing: taking quality of a hydrological parameter represented by hydrological related data contained in the marine environment data as attribute information of the marine environment data; obtaining a hydrological detection evaluation value based on the hydrological comprehensive interference condition and the hydrological parameter quality, and taking the hydrological detection evaluation value as the comprehensive quality evaluation value; selecting an adaptive data fusion mode based on a test precision of a hydrological parameter represented by the hydrological detection evaluation value.

[0012] In an optional embodiment, the method further includes: The target task scene is target identification of marine organisms in a marine environment, and the actual marine environment described by the scene description information is a comprehensive survival condition of marine organisms. The method further includes: For each type of marine environment data, the method further includes: obtaining a marine organism identification evaluation value based on the comprehensive survival condition of marine organisms and the target distinguishability, and taking the marine organism identification evaluation value as the comprehensive quality evaluation value; selecting an adaptive data fusion mode based on an identification precision of marine organisms represented by the marine organism identification evaluation value.

[0013] In an optional embodiment, the method further includes: The target task scene is abnormal early warning of marine organism density in a marine environment, and the actual marine environment described by the scene description information is a comprehensive survival condition of marine organisms. The method further includes: For each type of marine environment data, the method further includes: obtaining a marine organism density early warning evaluation value based on the comprehensive survival condition of marine organisms and the marine organism density credibility, and taking the marine organism density early warning evaluation value as the comprehensive quality evaluation value; selecting an adaptive data fusion mode based on a judgment precision of early warning represented by the marine organism density early warning evaluation value.

[0014] In an optional embodiment, the method further includes: The fusion environment data with multiple modalities is obtained by fusing the various types of marine environment data based on the data fusion mode, and the fusion environment data with multiple modalities comprises: obtaining a first fusion weight of each type of marine environment data under the data fusion mode in the target task scenario; When the data fusion mode is data-level fusion, the various types of marine environment data are aligned in data format, and then fused by using the corresponding first fusion weight to obtain the fusion environment data with multiple modalities; When the data fusion mode is feature-level fusion, the environment features of the various types of marine environment data are extracted respectively, and then the obtained environment features are unified in dimension, and then fused by using the corresponding first fusion weight to obtain the fusion environment data with multiple modalities.

[0015] In an optional embodiment, the method further comprises: When the data fusion mode is decision-level fusion, a second fusion weight of each type of marine environment data under the decision-level fusion mode in the target task scenario is obtained; The various types of marine environment data are analyzed respectively to obtain corresponding sub-detection results; Each sub-detection result is subjected to confidence evaluation to obtain a corresponding confidence evaluation result; Each sub-detection result whose confidence evaluation result meets a preset evaluation standard is regarded as a positive sub-detection result, and its corresponding second fusion weight is maintained; Each sub-detection result whose confidence evaluation result does not meet the preset evaluation standard is regarded as a negative sub-detection result, and its corresponding second fusion weight is reduced and adjusted to obtain an adjusted third fusion weight; Based on the positive sub-detection results and the corresponding second fusion weights, and in combination with the negative sub-detection results and the corresponding third fusion weights, sub-detection result fusion processing is performed to obtain a corresponding detection result.

[0016] In an optional embodiment, the method further comprises: The method is implemented by a target detection model, and after obtaining the detection result of the target task scenario, the method further comprises: The target task scenario, the various types of marine environment data and the detection result are updated to a sample data set as a new sample data, and a real environment state described by the detection result is taken as a sample label of the sample data; each other sample data in the sample data set comprises an other task scenario, various types of marine environment data collected in the other task scenario and a corresponding other detection result; The sample data set is divided into a training data set and a test data set; perform model training on the detection model to be trained based on the training data set, to obtain an initial detection model; perform test training on the initial detection model by using the test data set, to obtain the target detection model.

[0017] In an optional embodiment, the method further comprises: If the test accuracy of the target detection model does not reach a preset accuracy threshold, the method further comprises: filtering, from the test data set, each task scene corresponding to test data whose test result does not match a real environment state; For each task scene, the following are respectively performed: obtaining each type of new marine environment data under the task scene and a new detection result corresponding to the new marine environment data, and adding the task scene, the each type of new marine environment data and the new detection result as a piece of new sample data to the sample data set, and taking the real environment state described by the new detection result as a sample label of the new sample data; dividing, from the sample data set, a new training data set, and training the target detection model based on the new training data set, to obtain a new target detection model.

[0018] In a second aspect, the embodiments of the present application provide a detection device for a marine ecological environment, comprising: An acquisition module is configured to determine a target task scene, and acquire scene description information of the target task scene, wherein the scene description information is used to describe an actual marine environment under the target task scene, and acquire each type of marine environment data collected under the target task scene, wherein different types of marine environment data have different perception modalities. A processing module is configured to obtain a comprehensive quality evaluation value of the actual marine environment based on the scene description information and attribute information of the each type of marine environment data, and select a data fusion mode matched with the comprehensive quality evaluation value, and perform fusion processing on the each type of marine environment data based on the data fusion mode, to obtain fusion environment data with multiple modalities. An analysis module is configured to analyze the fusion environment data, to obtain a detection result of the target task scene.

[0019] In a third aspect, the present application provides an electronic device, comprising: A memory is configured to store a computer program. A processor is configured to execute the computer program stored in the memory, to implement the steps of the data updating method.

[0020] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the data updating method.

[0021] In the embodiments of the present application, the target task scene of marine ecological environment detection is determined first, and scene description information and multi-modal marine environment data are obtained. After low correlation data is removed through screening, the comprehensive quality evaluation value is obtained by combining the scene information and data attribute information. Then, the appropriate data fusion method is selected according to the comprehensive quality evaluation value. Subsequently, the data is fused according to the corresponding fusion method (data level, feature level) to obtain multi-modal fusion environment data. The detection result is output after analysis. The final detection result is obtained by fusing (decision level) the sub-detection results of various marine environment data.

[0022] At the same time, the sample data set is updated with the detection result and the real environment state, which is used for training and testing of the target detection model. If the accuracy of the model does not meet the standard, data will be supplemented for the non-matching scene and retrained, effectively avoiding the problem of insufficient detection comprehensiveness and poor task adaptation effect caused by independent calling of multi-modal data and inability to complement each other, and ensuring the detection accuracy and model adaptability. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 A structural schematic diagram of a system architecture provided by the embodiments of the present application is provided. Figure 2 A flowchart of a marine ecological environment detection method provided by the embodiments of the present application is provided. Figure 3 A method schematic diagram for obtaining various marine environment data provided by the embodiments of the present application is provided. Figure 4 A flowchart of a method for screening various marine environment data provided by the embodiments of the present application is provided. Figure 5 A method schematic diagram for screening various marine environment data provided by the embodiments of the present application is provided. Figure 6 A flowchart of a method for selecting a data fusion method for different target task scenes provided by the embodiments of the present application is provided. Figure 7 A flowchart of a method for data fusion of various marine environment data provided by the embodiments of the present application is provided. Figure 8 A flowchart of a method for fusing the detection results of various marine data provided by the embodiments of the present application is provided. Figure 9A A detection logic schematic diagram of a detection device for sensing equipment of marine ecological environment provided by the embodiments of the present application is provided. Figure 9BA detection device provided by an embodiment of the present application is a detection logic diagram of a marine ecological environment of a background device; Figure 10 A structural diagram of a detection device of a marine ecological environment provided by an embodiment of the present application is shown in the figure; Figure 11 A structural diagram of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. The specific operation method in the method embodiment can also be applied to the device embodiment or the system embodiment. It should be noted that in the description of the present application, “multiple” is understood as “at least two”. “And / or” describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. A and B are connected, which means that A and B are directly connected and A and B are connected through C. In addition, in the description of the present application, “first”, “second”, etc. are used only for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can it be understood as indicating or implying order.

[0025] With the continuous deepening of marine scientific research and the acceleration of marine resource development, the importance of marine ecological monitoring is increasingly prominent, and the current marine ecological monitoring system is developing towards intelligence and automation.

[0026] Under the related technology, in order to cope with the problem of limited detection ability and adaptation range of a single sensor, marine ecological monitoring has gradually introduced multi-sensor cooperation for marine ecological monitoring. By integrating multiple sensors (such as sonar, visible light camera and laser radar) in the underwater sensing terminal, multi-modal data (such as acoustic data, optical image data and three-dimensional point cloud data) of underwater targets are collected to obtain multi-dimensional information of targets under different water conditions, so as to make up for the limitations of single sensor in detection distance, imaging accuracy and environmental adaptability.

[0027] In the above scheme, each sensor collects corresponding modal data according to preset parameters, and then stores the collected multi-modal data simply. Subsequently, according to specific scene requirements (such as calling acoustic data in deep sea environment, calling optical image data in clear water body and calling laser radar data in shallow sea and medium depth water body), a certain modal data is selected, and then a monitoring task is performed relying on the modal data.

[0028] However, the above scheme only independently calls each modal data to adapt to the corresponding scene, and once the marine ecological detection faces dynamic changes in the environment or diversity tasks, the advantages of each modal data cannot be complementary.

[0029] To solve the above technical problems, in the embodiments of the present application, the target task scene is determined, the scene description information of the target task scene is obtained, and various types of marine environment data collected under the target task scene are obtained. Based on the scene description information and the attribute information of the various types of marine environment data, a comprehensive quality evaluation value of the actual marine environment is obtained, and a data fusion mode matched with the comprehensive quality evaluation value is selected. The various types of marine environment data are fused based on the data fusion mode, and the fusion environment data with multiple modalities are obtained. The fusion environment data is analyzed, and the detection result of the target task scene is obtained. Through this way, the problem of insufficient detection comprehensiveness and poor task adaptation effect caused by independent calling of multi-modal data and inability to complement each other can be avoided.

[0030] The system architecture diagram to which the technical scheme of the embodiments of the present application is applicable will be briefly introduced below. It should be noted that the system architecture diagram introduced below is only used to illustrate the embodiments of the present application and is not limited.

[0031] For example, refer to Figure 1 The system architecture diagram to which the embodiments of the present application are applicable is shown in FIG. 1. The system architecture at least includes a sonar 100, a visible light camera 101, a laser radar 102, and a marine backend processor 103.

[0032] Sonar 100: As an acoustic perception device for marine ecological environment detection, the sonar 100 is subjected to the turning force of water surface density, sea current, and terrain on the sound wave path, thereby generating acoustic echo data to detect the marine ecological environment; Visible light camera 101: As a visual perception device for marine ecological environment detection, the visible light camera 101 is subjected to the comprehensive modulation of sea surface illumination, water transparency, and atmospheric disturbance on the light path, thereby generating optical image data to detect the marine ecological environment in real time; Laser radar 102: As an optical perception device for marine ecological environment detection, the laser radar 102 is subjected to the scattering and absorption of sea surface waves, water optical properties, and aerosols on laser pulses, thereby generating three-dimensional point cloud data to detect the marine ecological environment in real time; Marine backend processor 103: As the core hub of marine ecological environment detection, the marine backend processor 103 is used to uniformly receive and process marine environment data from the sonar 100, the visible light camera 101, and the laser radar 102.

[0033] In addition, if the sonar 100, visible light camera 101, and lidar 102 have data analysis capabilities, they can each directly analyze the acquired acoustic data, optical image data, and 3D point cloud data to obtain the corresponding detection results. If the sonar 100, visible light camera 101, and lidar 102 do not have data analysis capabilities, the marine backend processor 103 processes and analyzes the acoustic data, optical image data, and 3D point cloud data uploaded by the sonar 100, visible light camera 101, and lidar 102 to obtain the detection results.

[0034] For example, see Figure 2 As shown in the embodiments of this application, the specific process of the marine ecological environment detection method is as follows: Step 200: Determine the target task scenario and obtain the scenario description information of the target task scenario.

[0035] Specifically, in this embodiment, the access sensing device is configured according to the target task, and the target task scenario is determined and the scenario description information of the target task scenario is obtained. The scenario description information describes the actual marine environment under the target task scenario.

[0036] For example, when shallow-sea organism detection is the target task scenario, the target task configuration integrates two sensing devices: lidar and visible light camera. When the scene description information under the target task scenario, that is, the actual marine environment does not have good lighting and water quality conditions, the data from both lidar and visible light camera sensing devices are fused to detect shallow-sea organisms.

[0037] By following the target task scenario and scenario description information, we can gain insight into the actual situation of the marine ecological environment, provide factual support for the dynamic configuration of sensing devices, and enable the task scenario-driven marine ecological environment detection method to have adaptive capabilities in the complex and ever-changing marine environment.

[0038] Step 201: Acquire various marine environmental data collected in the target mission scenario.

[0039] For details, please refer to Figure 3 As shown in the embodiments of this application, by configuring access sensing devices according to the target task scenario and using the sensing devices to collect marine environmental data, various types of marine environmental data can be obtained, and different types of marine environmental data have different sensing modalities.

[0040] For example, when the target task is to identify fish schools in the lower and middle layers of the water, sonar, visible light cameras, and lidar are specifically configured to address the characteristics of weak light and poor visible light penetration in these waters. Sonar can penetrate the water to obtain information on fish distribution, quantity, and size (acoustic data), but lacks color information; visible light cameras can identify fish density, morphology, and distinguish species by color (optical image data), but only cover the surface layer; lidar can construct the three-dimensional outline of fish schools, distinguish organisms, and correct sonar size errors (3D point cloud data). Based on these three sensing devices, acoustic data, optical image data, and 3D point cloud data of the marine environment can be collected, and the three types of marine environmental data have different sensing modalities, allowing for complementary advantages between different modalities.

[0041] By collecting and acquiring multimodal marine environmental data, we can capture multi-source information about the marine environment under different perception dimensions, providing data support for subsequent multimodal data fusion.

[0042] Furthermore, after performing step 201 and before performing step 202, in order to eliminate redundant data that is weakly related to the target task scenario and ensure that the data input to the target detection model is highly related to the target task, optionally, data filtering can also be performed on each type of marine environmental data.

[0043] For details, please refer to Figure 4 As shown in the embodiments of this application, the specific steps for filtering each type of marine environmental data are as follows: Step 4000: For each type of marine environmental data, perform the following: Delete the data in the marine environmental data whose correlation with the target task scenario does not meet the preset correlation conditions, and obtain the updated marine environmental data.

[0044] For details, please refer to Figure 5 As shown in the embodiments of this application, for each type of marine environmental data obtained, there may be some data whose correlation with the target task scenario does not meet the preset correlation conditions. In this case, it is necessary to delete this part of the data from each type of marine environmental data, so as to obtain marine environmental data with a high correlation with the target task scenario, that is, to obtain the updated marine environmental data.

[0045] For example, in the target task scenario of detecting coral reef bleaching, the data in the deep-water scattered echo data collected by sonar that is too far away from the coral reef will be removed, the images of coral reefs with yellowing due to clouds or obscured by ships collected by visible light cameras will be removed, and the non-coral reef point cloud data collected by lidar will be removed. Finally, the data that is highly correlated with the coral reef surface and its surrounding waters in each type of marine environmental data will be retained as the updated marine environmental data.

[0046] By eliminating the redundant information weakly related to the target task from each type of marine environment data, the subsequent processing load can be significantly reduced, and the updated marine environment data can be ensured to be consistent with the target task scene, thereby providing reliable data support for the subsequent execution of the target task.

[0047] In step 202, based on the scene description information and the attribute information of each type of marine environment data, a comprehensive quality evaluation value of the actual marine environment is obtained, and a data fusion mode matched with the comprehensive quality evaluation value is selected.

[0048] Specifically, in the embodiments of the present application, different target task scenes have different scene description information, and the attribute information of each type of marine environment data obtained under different target task scenes is different. The attribute information of each type of marine environment data describes the comprehensive quality of each type of marine environment data in the corresponding task scene. Based on the scene description information and the attribute information of each type of marine environment data, a comprehensive quality evaluation value of the actual marine environment can be obtained, and based on the comprehensive quality evaluation value, a data fusion mode matched therewith can be selected.

[0049] Specifically, referring to FIG. 2, for different target task scenes, different data fusion modes are selected, and the technical process of selecting the data fusion mode also has differences, which can include the following operations: Figure 6 In step 202-1, the target task scene is to detect the hydrological parameters of the marine environment, the actual marine environment described by the scene description information is the hydrological comprehensive interference condition, and based on the scene description information and the attribute information of each type of marine environment data, a comprehensive quality evaluation value of the actual marine environment is obtained, and a data fusion mode matched with the comprehensive quality evaluation value is selected.

[0050] Specifically, for each type of marine environment data, the quality of the hydrological parameters represented by the hydrological related data contained in the marine environment data is taken as the attribute information of the marine environment data. Then, based on the hydrological comprehensive interference condition and the quality of the hydrological parameters, a corresponding hydrological detection evaluation value is obtained, and the hydrological detection evaluation value is taken as the comprehensive quality evaluation value. Finally, based on the test accuracy of the hydrological parameters represented by the hydrological detection evaluation value, an appropriate data fusion mode is selected.

[0051] ​For example, in the task of detecting hydrological parameters in a near-shore aquaculture area (the core requirement of the task is to detect flow rate, sediment concentration, and water depth to support the prevention of water flow impact on aquaculture organisms, the prevention and control of sediment deposition, and the adaptation of net cage layout), first, considering the comprehensive hydrological disturbance of the sea area near the estuary, such as the turbulent water flow during the rising tide, the sudden increase in sediment concentration, the smooth water during the ebbing tide, and the fluctuation of light, configure sonar, visible light camera, and laser radar to collect various types of marine environmental data, and according to the target task, set the sonar weight to 0.6 (sonar obtains flow rate data through acoustic Doppler principle, can monitor middle and lower layer flow rate but has error due to turbulent flow, medium quality of hydrological parameters), visible light camera weight to 0.5 (camera calculates sediment concentration through image gray scale contrast, can capture surface sediment concentration change but is greatly affected by light interference, medium to low quality of hydrological parameters), and laser radar weight to 0.8 (laser radar obtains water depth data through laser ranging, has little influence from water transparency and stable precision, higher quality of hydrological parameters).

[0052] Then, the flow rate data collected by the sonar is used to pre-judge whether the water flow will impact the aquaculture organisms, the sediment concentration data collected by the visible light camera is used to evaluate the risk of sediment deposition, and the water depth data collected by the laser radar is used to adapt the layout of the aquaculture net cage. According to the collected parameter quality of the hydrological data, set the flow rate data weight to 0.4 (related to the core requirement of aquaculture organism safety prevention), the sediment concentration data weight to 0.3 (related to the requirement of sediment deposition prevention and control in the aquaculture area), and the water depth data weight to 0.3 (related to the requirement of precise layout of the aquaculture net cage), then the hydrological detection evaluation value is 0.6x0.4+0.5x0.3+0.8x0.3, i.e. the hydrological detection evaluation value is 0.7. At this time, the hydrological detection evaluation value represents the precision P of the hydrological parameter detection as 0.7.

[0053] According to the task configuration of the hydrological parameter detection, set the device computing power level C to 0.8, set the environmental dynamics E to 0.2 considering the significant rising tide disturbance but stable ebbing tide of the hydrological environment, and set the real-time requirement L to 0.1 based on the requirement of generating a hydrological detection report once a day.

[0054] The sensitivity of different data fusion methods to precision requirement P, device computing power level C, environmental dynamics E, and real-time requirement L is different, and the preset weights of the three data fusion methods to precision requirement P, device computing power level C, environmental dynamics E, and real-time requirement L are shown in Table 1: Table 1

[0055] The system calculates the adaptation score of each fusion method through the weighted scoring formula: According to the preset precision weight of each data fusion method in Table 1 , computing power weight environmental dynamics weight and real-time weight and the actual precision P, the device computing power level C, the environmental dynamics E and the real-time requirement L in the hydrological parameter detection scene, the data-level fusion score can be obtained is 1.32, the feature-level fusion score is 1.11, and the decision-level fusion score is 0.78. Therefore, when the hydrological parameter detection task is performed on the near-shore aquaculture area, the selected data fusion manner is data-level fusion.

[0056] Step 202-2, the target task scene is: target identification is performed on marine organisms in the marine environment, the actual marine environment described by the scene description information is: the comprehensive survival condition of marine organisms; based on the scene description information and the attribute information of each type of marine environment data, a comprehensive quality evaluation value of the actual marine environment is obtained, and a data fusion manner matched with the comprehensive quality evaluation value is selected.

[0057] Specifically, in the examples of the present application, for each type of marine environment data, the following is performed: the target distinguishability represented by the marine organism related data contained in the marine environment data is taken as the attribute information of the marine environment data; then, based on the comprehensive survival condition of marine organisms and the target distinguishability, a corresponding marine organism identification evaluation value is obtained, and the marine organism identification evaluation value is taken as the comprehensive quality evaluation value; finally, based on the identification accuracy of marine organisms represented by the marine organism identification evaluation value, an adaptive data fusion manner is selected.

[0058] For example, in the target identification task of marine organisms in the sea area around the near-shore aquaculture area (the core requirement of the task is to identify the species and quantity of fish groups, the density of plankton and the position of jellyfish, so as to support the evaluation of bait resources, water quality monitoring and enemy and pest control in the aquaculture area), first, according to the comprehensive survival condition of marine organisms in the sea area, such as the concentration of fish groups in the lower layer, the light sensitivity of plankton and the tidal activity of jellyfish, sonar, visible light camera and laser radar are configured to collect various types of marine environment data, and according to the dependence of the target task on the data of each device, the weight of the sonar is set to 0.6 (the sonar obtains the two-dimensional contour and echo intensity of the fish group, which can preliminarily identify the distribution of the fish group but lacks color information, and the target distinguishability is medium), the weight of the visible light camera is set to 0.55 (the camera captures the color and shape of the surface layer organism, which can distinguish plankton and jellyfish but is greatly affected by light, and the target distinguishability is medium to low), and the weight of the laser radar is set to 0.9 (the laser radar generates a three-dimensional contour of the organism, which can distinguish organisms and non-organisms and correct size misjudgment, and the target distinguishability is high).

[0059] Then, the fish school acoustic data collected by the sonar is used to preliminarily judge the distribution and quantity of the fish school, the surface biological optical image data collected by the visible light camera is used to identify the density of plankton and the shape of jellyfish, and the biological three-dimensional point cloud data collected by the laser radar is used to construct the three-dimensional contour of the biological and exclude non-biological interference. Then, according to the target distinguishability of the collected marine biological data, the fish school acoustic data weight is set to 0.4 (related to the core demand of bait resource evaluation), the surface biological optical image data weight is set to 0.3 (related to the demand of water quality and enemy control), and the biological three-dimensional point cloud data weight is set to 0.3 (related to the demand of identification accuracy), so that the marine biological identification evaluation value 0.6x0.4+0.55x0.3+0.9x0.3, that is, the marine biological identification evaluation value is 0.675. At this time, the marine biological identification evaluation value represents the accuracy P of target identification of marine biology is 0.675.

[0060] Finally, according to the task configuration of marine biological identification, considering that the device computing power level C is set to 0.6 for the on-site deployment of the ship-mounted middle-end processor, the environmental dynamics E is set to 0.5 considering the change of biological distribution with light and tide, and the real-time requirement L is set to 0.4 based on the demand of updating the identification result every hour. Combined with the related preset weights in Table 1, the data level fusion score is 1.2275, the feature level fusion score is 1.2425, and the decision level fusion score is 1.22. Therefore, for the marine biological target identification task of the sea area around the nearshore aquaculture area, the selected data fusion mode is feature level fusion.

[0061] Step 202-3, the target task scene is: abnormal early warning of marine biological density in marine environment, the actual marine environment described by the scene description information is: comprehensive survival condition of marine biology; based on the scene description information and the attribute information of each type of marine environment data, the comprehensive quality evaluation value of the actual marine environment is obtained, and the data fusion mode matched with the comprehensive quality evaluation value is selected, Specifically, in the example of the present application, for each type of marine environment data, the marine biological density credibility represented by the marine biological related data contained in the marine environment data is taken as the attribute information of the marine environment data. Then, based on the comprehensive survival condition of marine biology and the marine biological density credibility, the marine biological density early warning evaluation value is obtained, and the marine biological density early warning evaluation value is taken as the comprehensive quality evaluation value. Finally, based on the early warning judgment accuracy represented by the marine biological density early warning evaluation value, the adaptive data fusion mode is selected.

[0062] For example, in the abnormal early warning task of marine biological density in the sea area around the near-shore breeding area (the core requirement of the task is to monitor the density changes of plankton and fish, and to trigger an early warning when the density of plankton exceeds the preset plankton density threshold and the density of fish is lower than the preset fish density threshold, so as to prevent red tide disasters and ensure the supply of bait for breeding organisms), first, combined with the tidal fluctuations of plankton in the sea area, the aggregation distribution of fish, and the comprehensive survival conditions of impurity confused marine organisms, the sonar, visible light camera and laser radar are configured to collect various types of marine environmental data, and according to the dependence of the target task on each device data, the sonar weight is set to 0.65 (the sonar can penetrate the water to monitor the middle and lower layer fish by inverting the fish density through the echo intensity, but the density calculation credibility is medium due to the interference of impurity echo), the visible light camera weight is set to 0.5 (the camera can capture the surface layer density change by counting the pixel proportion of the image to count the plankton density, but the data credibility is medium to low due to the influence of light reflection), and the laser radar weight is set to 0.85 (the laser radar can distinguish between biological and non-biological by point cloud density, can exclude impurity interference, and can accurately count the biological density, and the data credibility is higher).

[0063] Then, the fish acoustic data collected by the sonar is used to invert the middle and lower layer fish density, the surface layer biological optical image data collected by the visible light camera is used to count the plankton density, and the biological three-dimensional point cloud data collected by the laser radar is used to distinguish between biological and non-biological and correct the density statistical error. Then, according to the density credibility of the collected marine biological data, the fish acoustic data weight is set to 0.4 (related to the core requirement of fish density early warning and bait supply guarantee), the surface layer biological optical image data weight is set to 0.4 (related to the core requirement of plankton red tide early warning), and the biological three-dimensional point cloud data weight is set to 0.2 (related to the requirement of removing interference from density data and improving early warning accuracy), then the marine biological density early warning evaluation value 0.65x0.4+0.5x0.4+0.85x0.2 can be obtained, i.e. the marine biological density early warning evaluation value is 0.61. At this time, the marine biological density early warning evaluation value represents the accuracy P of the marine biological density abnormal early warning as 0.61.

[0064] Finally, according to the task configuration of marine biological density early warning, considering the on-site deployment of shipboard embedded processor (only supporting lightweight data operation), the device computing power level C is set to 0.4, combined with the dynamic change of biological density with the tide, the environmental dynamics E is set to 0.8 (low environmental stability), and based on the requirement of the task to output early warning results in a short time, the real-time requirement L is set to 0.9 (high real-time requirement). Combined with the related preset weights in Table 1, the data level fusion score is 1.119, the feature level fusion score is 1.427, and the decision level fusion score The value is 1.804. Therefore, for the early warning task of abnormal marine organism density in the waters surrounding nearshore aquaculture areas, the selected data fusion method is decision-level fusion.

[0065] By combining scenario descriptions of different target tasks with attribute information of various marine environmental data in the corresponding scenarios (such as hydrological parameter quality, target identifiability, and biological density reliability), a comprehensive quality assessment value of the actual marine environment is obtained. Based on the accuracy requirements reflected by the assessment value, and combined with the requirements of equipment computing power, environmental dynamics, and real-time performance, an appropriate data fusion method is selected. This approach can specifically match the core needs of different detection tasks, fully leverage the synergistic advantages of multimodal data, avoid detection bias caused by mismatch between fusion methods and tasks, and effectively improve the accuracy and adaptability of marine ecological environment detection.

[0066] Step 203: Based on the data fusion method, perform fusion processing on various types of marine environmental data to obtain fused environmental data with multiple modes.

[0067] For details, please refer to Figure 7 As shown, the process of fusing various types of marine environmental data to obtain multimodal fused environmental data includes the following technical steps: Step 2030: Obtain the first fusion weight of each type of marine environmental data under the data fusion method in the target task scenario.

[0068] Specifically, in this embodiment of the application, the weight of each type of marine environmental data obtained under the target mission scenario is different. Therefore, when performing data fusion, it is necessary to obtain the first fusion weight of each type of marine environmental data under the data fusion method. The setting of the first fusion weight needs to be combined with the core requirements of the target mission, the contribution of each type of data in the fusion method, and the actual data quality to ensure that the weight allocation is highly adapted to the mission objectives.

[0069] Step 2030-1: When the data fusion method is data-level fusion, after aligning the data formats of various marine environmental data, the corresponding first fusion weight is used for fusion to obtain fused environmental data with multiple modes.

[0070] Specifically, in this embodiment of the application, if a data-level fusion method is selected for the target task scenario, the sensing device will align the formats of the various types of marine data collected, and then fuse the various types of marine data that have been format aligned according to the first fusion weight of each modal data, thereby obtaining fused environmental data with multiple modalities.

[0071] For example, in the target scene of step 202-1 for hydrological parameter detection of a near-shore aquaculture area, the selected data level fusion mode is determined, and the first fusion weights of the flow velocity data collected by the sonar, the sediment concentration data in water collected by the visible light camera, and the water depth data collected by the laser radar are 0.4, 0.3, and 0.3 respectively. First, the formats of the three types of hydrological data are aligned, and are uniformly converted into a structured data format containing a timestamp, a spatial coordinate, and a parameter value; then, the aligned data is weighted and calculated according to the first fusion weights. For example, for the hydrological data at 10:00:00 (a time) and north latitude 30°15', east longitude 120°20' (a spatial coordinate), the aligned sonar flow velocity data is 10:00:00, north latitude 30°15', east longitude 120°20', 0.3 m / s, the aligned visible light camera sediment concentration data is 10:00:00, north latitude 30°15', east longitude 120°20', 0.2 kg / m³, and the aligned laser radar water depth data is 10:00:00, north latitude 30°15', east longitude 120°20', 8.1 m; then, the aligned data at the same space-time is weighted and calculated according to the first fusion weights, so that the fusion value of the flow velocity at the space-time of 10:00:00, north latitude 30°15', east longitude 120°20' is 0.3*0.35, the fusion value of the sediment concentration in water is 0.2*0.25, and the fusion value of the water depth is 8.1*0.4, and finally the fusion environmental data of the space-time point is obtained, that is, the flow velocity is 0.12 m / s, the water depth sediment concentration is 0.06 kg / m³, and the water depth is 2.43 m.

[0072] Step 2030-2, when the data fusion mode is feature level fusion, the environmental features of each type of marine environmental data are extracted respectively, and after the obtained environmental features are unified in dimension, the corresponding first fusion weights are used for fusion to obtain fusion environmental data with multiple modalities.

[0073] Specifically, in the embodiments of the present application, if the feature level fusion mode is selected for the target task scene, the environmental features capable of representing the core requirements of the task are first extracted from each type of marine environmental data, and then the environmental features with unified feature dimensions are fused to obtain fusion environmental data with multiple modalities.

[0074] For example, in the target scene of identifying marine biological targets in the near-shore breeding area in step 202-2, the selected feature-level fusion mode is determined, and the first fusion weights of the fish acoustic data collected by the sonar, the surface biological optical image data collected by the visible light camera, and the biological three-dimensional point cloud data collected by the laser radar are 0.4, 0.3, and 0.3 respectively. First, the environmental features of the three types of data are extracted respectively: the fish school echo intensity mean, target contour aspect ratio and other features (used to represent the fish school distribution density and body shape) are extracted from the sonar fish school acoustic data, the color histogram features, biological contour edge gradient and other features (used to distinguish the morphological color difference between plankton and jellyfish) are extracted from the visible light camera surface biological optical image data, and the point cloud density, target three-dimensional volume proportion and other features (used to exclude non-biological interference and correct biological body shape judgment) are extracted from the laser radar biological three-dimensional point cloud data; then, the obtained environmental features are processed for dimension unification, the sonar echo intensity mean (0-255), the visible light color histogram feature (0-1), and the laser radar point cloud density (0-100 points / cm2) are uniformly mapped to the feature dimension of 0-1 (such as the sonar echo intensity mean 200 normalized to 0.78, the visible light color feature 0.6 remains unchanged, and the laser radar point cloud density 50 points / cm2 normalized to 0.5), to ensure that different modal features can be directly involved in fusion calculation; then, the dimension-unified same space-time point feature data is calculated according to the first fusion weight. For example, for the space-time point of 10:30:00, north latitude 30°16', east longitude 120°21', the dimension-unified feature data is: sonar fish school feature value 0.6, visible light surface biological feature value 0.55, and laser radar three-dimensional feature value 0.8, which are calculated respectively with the corresponding weight to obtain the fish school feature fusion value 0.6x0.4, the surface biological feature fusion value 0.55x0.35, and the three-dimensional feature fusion value 0.8x0.25. Finally, the fusion environmental data of the space-time point is obtained as: fish school feature value 0.24, surface biological feature value 0.1925, and three-dimensional feature value 0.2.

[0075] By obtaining the first fusion weight of each type of marine environmental data under the corresponding data fusion mode for different target task scenes, and then processing according to the difference in fusion mode, the data format is calculated by weighting according to the first fusion weight in data-level fusion, and the environmental features of each type of data are extracted and dimension-unified in feature-level fusion, and then calculated by weighting according to the first fusion weight, finally the fusion processing of each type of marine environmental data can be completed, and the multi-modal fusion environmental data suitable for task requirements can be obtained.

[0076] In step 204, the fusion environmental data is analyzed to obtain the detection result of the target task scene.

[0077] Specifically, in the embodiments of the present application, after various types of marine environment data are processed by corresponding fusion methods to obtain multi-modal fusion environment data, the adaptive analysis model and judgment rule are called in combination with the core requirements of the task, the key features in the fusion data are extracted and matched with the classification standard, the interference information is removed, and the final detection result is determined.

[0078] For example, in step 202-3, assuming that the fusion data of a certain space-time point (11:00:00, 30°15'N, 120°20'E) obtained by decision-level fusion is: fish density fusion value 0.3 tail / m³, plankton density fusion value 0.4 kg / m³. Call the biological density early warning analysis model, extract the key features of fish / plankton density, match the preset standard (fish density <0.5 tail / m³ needs to be warned, plankton density >0.5 kg / m³ needs to be warned) and determine: the fish density of this space-time point is low and needs to pay attention to bait replenishment, and the plankton density does not reach the warning threshold. Finally, the detection result of fish density warning and plankton safety in the sea area at 11:00:00 is output, which meets the density anomaly warning requirement.

[0079] Through the analysis of the fusion environment data, the adaptive analysis model and judgment rule are called in combination with the core requirements of the task, the key features are accurately extracted and matched with the classification standard, and the detection result of the target task scene is efficiently obtained. This process can meet different task requirements, ensure the accuracy and reliability of the detection result, provide a strong basis for subsequent decision-making, and effectively improve the overall efficiency of marine ecological environment detection.

[0080] In an optional embodiment, when the data fusion method is decision-level fusion, the sub-detection results of various types of marine data are fused to obtain the final detection result.

[0081] Specifically, referring to Figure 8 In the embodiments of the present application, when the data fusion method is decision-level fusion, the second fusion weight of each type of marine environment data under the decision-level fusion method in the target task scene is obtained. Then, each type of marine environment data is analyzed to obtain the corresponding sub-detection result, and the confidence evaluation result of each sub-detection result is obtained. Then, each sub-detection result whose confidence evaluation result meets the preset evaluation standard is taken as a positive sub-detection result, and its corresponding second fusion weight is kept. Each sub-detection result whose confidence evaluation result does not meet the preset evaluation standard is taken as a negative sub-detection result, and its corresponding second fusion weight is reduced to obtain the adjusted third fusion weight. Finally, based on each positive sub-detection result and the corresponding second fusion weight, in combination with each negative detection result and the corresponding third fusion weight, the sub-detection result fusion processing is performed to obtain the corresponding detection result.

[0082] For example, in the target task scenario of abnormal marine biological density early warning for near-shore breeding area in step 202-3, the selected decision-level fusion mode is as follows. First, the second fusion weights of sonar (fish density), visible light camera (phytoplankton density) and laser radar (non-biological interference) data are obtained, which are 0.4, 0.4 and 0.2 respectively. Then, the data of each type is analyzed. The acoustic data fish density is 0.3 tail / m³ (lower than the early warning threshold of 0.5 tail / m³, which needs to be warned), the optical image data phytoplankton density is 0.6 kg / m³ (higher than the early warning threshold of 0.5 kg / m³, which needs to be warned), and the three-dimensional point cloud data non-biological impurity proportion is 5% (no interference), and three sub-detection results are obtained. Subsequently, the confidence is evaluated. The sonar result confidence is 0.85, the visible light result confidence is 0.6, and the laser radar result confidence is 0.9. Then, the positive / negative sub-detection results are determined. The sonar and laser radar results meet the standard and are positive detection sub-results, and the second fusion weights are kept. The visible light result does not meet the preset evaluation standard and is a negative detection sub-result, and its second fusion weight is reduced from 0.4 to 0.2 (third fusion weight). Finally, the fused confidence is obtained by 0.4 x 0.85 + 0.2 x 0.6 + 0.2 x 0.9 = 0.52, and the detection result is that the fish density in the near-shore breeding area at a certain time and space is abnormal and needs to be closely warned, and the phytoplankton density is abnormal and needs to be carefully concerned.

[0083] By decision-level fusion of sub-detection results of various types of marine environment data and adjusting the weight according to the confidence evaluation, the influence of high-confidence results can be highlighted and the interference of low-confidence information can be weakened. This way can integrate the judgments of multi-source data, improve the reliability and accuracy of the final detection result, effectively integrate the advantages of different types of data, provide more comprehensive and accurate detection conclusions for the target task scenario, and meet the effective fusion and efficient analysis needs of complex data in marine environment detection.

[0084] In an optional embodiment, the detection method of the marine ecological environment is realized by a target detection model. After obtaining the detection result of the target task scenario, the target detection model is trained, Specifically, in the embodiments of the present application, first, the target task scene, various types of marine environment data and detection results are updated as a new sample data to the sample data set, and the real environment state described by the detection result is taken as the sample label of the sample data. Among them, each other sample data in the sample data set contains: one other task scene, various types of marine environment data collected in the other task scene and the corresponding other detection result; then, the sample data set is divided into a training data set and a test data set, the detection model to be trained is trained based on the training data set, and an initial detection model is obtained. The test data set is used to test and train the initial detection model, and a target detection model is obtained.

[0085] For example, for detecting offshore plankton to master the marine ecological situation, first, the target task scene is determined as the summer offshore plankton monitoring area, various types of marine environment data such as seawater temperature, salinity data and plankton image in the area are collected through a sensor, and a preliminary detection tool is used to obtain a detection result, which shows that the plankton density in the area is 280 per liter, and contains 3 dominant species. Then, the summer offshore plankton monitoring area, the collected various types of marine environment data and the above detection result are taken as new sample data, and are updated to the sample data set, and the real environment state of the plankton density in the area per liter containing how many dominant species is taken as the sample label, while other samples in the sample data set contain other scenes such as "spring offshore plankton monitoring area", corresponding environment data and detection results. Then, the sample data set is divided into training and test data sets, the detection model to be trained is trained using the training data set, an initial detection model is obtained, and the test data set is used for test training, and finally a target detection model capable of accurately detecting offshore plankton is obtained.

[0086] By integrating the target task scene, related marine environment data and detection results into new samples, updating the data set with real environment states as labels, and then training and testing the process to build the target detection model, the sample diversity and scene coverage of model learning can be continuously enriched. This way can make the model continuously absorb new scene information, improve the adaptability and detection accuracy of different task scenes, and ensure that the model output result is highly consistent with the actual environment state, providing reliable model support for marine ecological environment detection.

[0087] In an optional embodiment, if the test accuracy of the target detection model does not reach the preset accuracy threshold, the target detection model is retrained.

[0088] Specifically, in the embodiments of the present application, each task scene corresponding to test data whose test result does not match the real environment state is screened out from the test data set; for each task scene, the following are respectively performed: obtaining each type of new marine environment data under the task scene and the corresponding new detection result, and adding the task scene, each type of new marine environment data and the new detection result as a new sample data to the sample data set, and taking the real environment state described by the new detection result as the sample label of the new sample data; dividing the new training data set from the sample data set, training the target detection model based on the new training data set, and obtaining the new target detection model.

[0089] It should be noted that before obtaining each type of new marine environment data under the task scene and the corresponding new detection result, the test data whose test result does not match the real environment state is screened out from the test data set for data cleaning and enhancement processing to make the data quality up to standard, and then it is judged whether the sample data in the existing sample data set is sufficient. If the sample data is insufficient, the data collection is triggered to supplement the sample data set (i.e., obtaining each type of new marine environment data under the task scene and the corresponding new detection result).

[0090] For example, in the test of the near-shore plankton detection model, the accuracy does not reach the preset threshold, then the scenes whose detection result does not match the real state are screened out from the test data, and the test data of these scenes is processed: the fuzzy plankton image is removed, the abnormal sea water temperature value is corrected, and the image is rotated and brightness adjusted to enhance the data quality. For example, if the sample quantity under the existing scene is only 12, which is far from meeting the demand, then new data collection is started to supplement the sea water monitoring data and high-definition plankton image for 7 consecutive days in the region. Then, the new data under the target scene and the real state label are added to the sample set, the original model is retrained after the training set is redivided. Finally, the new target detection model is obtained.

[0091] Through the data cleaning and enhancement, sample supplement and retraining for the task scenes that do not match in the model test, the detection short board of the model in the specific scene can be accurately solved. This way can targetedly improve the sample quality and quantity, and strengthen the adaptation ability of the model to complex scenes, so that the accuracy of the retrained model is effectively improved and reaches the preset threshold, ensuring that it continuously outputs reliable results in marine ecological environment detection, and enhancing the practicality and stability of the model.

[0092] In the above embodiments, the process of obtaining, processing and analyzing each type of marine environment data is completed by the detection device, and the detection device can correspond to different functional entities in different application scenarios.

[0093] On the one hand, when the sensing device has data analysis capabilities, it can be used directly as a detection device. When placed in seawater, the sensing device can collect multimodal marine environmental data in real time, including acoustic data, optical image data, and 3D point cloud data, using its built-in sensors. Furthermore, it can analyze and generate corresponding detection results directly based on the collected marine environmental data, tailored to the specific task scenario, without relying on external equipment for analysis.

[0094] For example, see Figure 9A As shown, the multimodal perception layer 900 resides on the sensing device, while the global data processing layer 901, the integrated training and push central layer 902, and the intelligent interactive application layer 903 reside on the backend device. When the sensing device has data parsing capabilities, it uploads the obtained detection results to the integrated training and push central layer 902 via the multi-source data access gateway of the global data processing layer 901. The integrated training and push central layer 902 analyzes the received test results according to the configured business logic, generates the analysis results for the final target task, and then uploads the analysis results to the intelligent interactive layer 903 for real-time display. If the current scene model training task requires new data, the sensing device uploads various types of marine environmental data collected to the global data governance layer 901. Then, according to the scene task configuration, it processes the various types of marine environmental data into training data and adds them to the sample dataset. After the training dataset is partitioned, it is uploaded to the integrated training and push central layer 902 for training the new scene model.

[0095] For example, in the target scenario of detecting shallow-sea organisms, this task is equipped with two modes of sensing devices: LiDAR and visible light cameras. An environmental detection model can be trained based on the collected optical image data to determine underwater video acquisition or target detection. The model outputs three categories: 1) Good environment: refers to an underwater environment with good lighting and water quality, where the optical image data quality is good. Target detection using only optical image data yields relatively accurate results; that is, only video images captured by the optical camera are used to call the corresponding model for real-time analysis, saving computing power; 2) Moderate environment: Detection is performed by fusing the LiDAR and visible light camera models. The advantage of LiDAR in detecting targets with poor visibility is utilized to improve detection accuracy. Appropriate image enhancement processing can be enabled for the optical image data; 3) Poor environment: Optical image data quality is too poor to be recovered through image enhancement. Analysis can be performed using only the 3D point cloud detection model of the LiDAR. Environmental conditions can be determined using a lightweight image quality assessment method with strong real-time performance. Assessment parameters include: a) Brightness L: the average pixel value of the image; b) Contrast C: the standard deviation of the image pixels. The judgment logic is as follows: ① Good environment: L≥120 and C≥40; ② Average environment: 80≤L<120 or 20≤C<40; ③ Poor environment: L<80 or C<20. The evaluation method, parameters, and thresholds can be defined and adjusted in the target task scenario configuration.

[0096] On the other hand, when the sensing devices lack data analysis capabilities, the detection devices act as back-end equipment. In this case, the sensing devices deployed in seawater are only responsible for collecting various types of marine environmental data, such as acoustic data, optical image data, and 3D point cloud data. After collection, these raw data are uploaded to the back-end equipment located at sea, where the back-end equipment performs unified data analysis and processing to obtain the detection results for the target task scenario.

[0097] For example, see Figure 9B As shown, when the sensing device lacks data parsing capabilities, after collecting raw marine environmental data such as acoustic data, optical image data, and 3D point cloud data, the sensing device directly uploads it to the multi-source data access gateway of the full-domain data processing layer 901. The gateway performs data format verification and preliminary straightening. Subsequently, the full-domain data processing layer 901 transmits the processed raw data to the training and propulsion integrated central layer 902. This layer calls the appropriate parsing model to analyze the data, generate detection results for the target task scenario, and then optimizes them according to business logic before uploading them to the intelligent interactive application layer 903 for display. If additional model training data is needed, the full-domain data processing layer 901 will filter data that meets the scenario requirements from the stored raw data, trigger the intelligent preprocessing module to convert it into training data and integrate it into the sample dataset. After dividing the training dataset, it is uploaded to the training and propulsion integrated central layer 902 for training the model in the new scenario.

[0098] The detection device's acquisition, processing, and analysis modules work together, combining scene information and data attributes to select an appropriate fusion method. It can also update the sample training model. Furthermore, the detection device can correspond to different functional entities (sensing devices with analytical capabilities or back-end devices). It can enable sensing devices to directly generate detection results or have back-end devices process the data, thus fully adapting to the needs of multiple scenarios and improving the accuracy, flexibility, and model adaptability of marine ecological environment detection.

[0099] Based on the same inventive concept, this application also provides a marine ecological environment detection device, for example, see [reference]. Figure 10 As shown, the marine ecological environment monitoring device 1000 includes: Acquisition Module 10000: Used to determine the target task scenario and acquire the scenario description information of the target task scenario; the scenario description information is used to describe the actual marine environment under the target task scenario, and to acquire various types of marine environmental data collected under the target task scenario; among them, different types of marine environmental data have different perception modalities; Processing module 10001: is used to obtain the comprehensive quality assessment value of the actual marine environment based on scene description information and attribute information of various marine environmental data, and select a data fusion method that matches the comprehensive quality assessment value; and to perform fusion processing on various marine environmental data based on the data fusion method to obtain fused environmental data with multiple modes; Parsing module 10002: Parses the fused environment data to obtain the detection results of the target task scene.

[0100] In an optional embodiment, the processing module 10001 is specifically used for: After acquiring various marine environmental data collected in the target mission scenario, and before obtaining the comprehensive quality assessment value of the actual marine environment based on the scenario description information and the attribute information of various marine environmental data, the following steps are also included: For each type of marine environmental data, the following steps are performed: delete the data from the marine environmental data that does not meet the preset correlation conditions with the target task scenario, and obtain the updated marine environmental data.

[0101] In an optional embodiment, the processing module 10001 is further configured to: When the target task scenario is: to detect hydrological parameters of the marine environment, the actual marine environment described by the scenario description information is: comprehensive hydrological disturbances. Based on scene description information and attribute information from various marine environmental data, a comprehensive quality assessment value for the actual marine environment is obtained, and a data fusion method matching the comprehensive quality assessment value is selected, including: For each type of marine environment data, the following is performed: taking the quality of the hydrological parameter represented by the hydrological related data contained in the marine environment data as the attribute information of the marine environment data; Based on the hydrological comprehensive interference condition and the quality of the hydrological parameter, a corresponding hydrological detection evaluation value is obtained, and the hydrological detection evaluation value is taken as the comprehensive quality evaluation value; Based on the test accuracy of the hydrological parameter represented by the hydrological detection evaluation value, an adaptive data fusion mode is selected.

[0102] In an optional embodiment, the processing module 10001 is further configured to: The target task scenario is to perform target identification on marine organisms in the marine environment, and the actual marine environment described by the scene description information is the comprehensive survival condition of marine organisms; Based on the scene description information and the attribute information of each type of marine environment data, a comprehensive quality evaluation value of the actual marine environment is obtained, and a data fusion mode matched with the comprehensive quality evaluation value is selected, including: For each type of marine environment data, the following is performed: taking the target distinguishability represented by the marine organism related data contained in the marine environment data as the attribute information of the marine environment data; Based on the comprehensive survival condition of marine organisms and the target distinguishability, a corresponding marine organism identification evaluation value is obtained, and the marine organism identification evaluation value is taken as the comprehensive quality evaluation value; Based on the identification accuracy of the marine organism represented by the marine organism identification evaluation value, an adaptive data fusion mode is selected.

[0103] In an optional embodiment, the processing module 10001 is further configured to: The target task scenario is to perform abnormal early warning on the density of marine organisms in the marine environment, and the actual marine environment described by the scene description information is the comprehensive survival condition of marine organisms; Based on the scene description information and the attribute information of each type of marine environment data, a comprehensive quality evaluation value of the actual marine environment is obtained, and a data fusion mode matched with the comprehensive quality evaluation value is selected, including: For each type of marine environment data, the following is performed: taking the marine organism density reliability represented by the marine organism related data contained in the marine environment data as the attribute information of the marine environment data; Based on the comprehensive survival condition of marine organisms and the marine organism density reliability, a marine organism density early warning evaluation value is obtained, and the marine organism density early warning evaluation value is taken as the comprehensive quality evaluation value; Based on the early warning judgment accuracy represented by the marine organism density early warning evaluation value, an adaptive data fusion mode is selected.

[0104] In an optional embodiment, the processing module 10001 is further configured to: The various types of marine environment data are fused based on a data fusion manner to obtain fusion environment data with multiple modalities, including: obtaining first fusion weights of the various types of marine environment data under the data fusion manner respectively under a target task scenario; When the data fusion manner is data-level fusion, the various types of marine environment data are aligned in data format, and then fused using the corresponding first fusion weights to obtain fusion environment data with multiple modalities; When the data fusion manner is feature-level fusion, the environment features of the various types of marine environment data are extracted respectively, and then the obtained environment features are unified in dimension, and then fused using the corresponding first fusion weights to obtain fusion environment data with multiple modalities.

[0105] In an optional embodiment, the processing module 10001 is further configured to: After obtaining the detection result of the target task scenario, the method further includes: updating the target task scenario, the various types of marine environment data and the detection result as a new sample data to the sample data set, and taking the real environment state described by the detection result as a sample label of the sample data; each other sample data in the sample data set includes: an other task scenario, various types of marine environment data collected under the other task scenario and a corresponding other detection result; dividing the sample data set into a training data set and a test data set; training the detection model to be trained based on the training data set to obtain an initial detection model; testing and training the initial detection model using the test data set to obtain a target detection model.

[0106] In an optional embodiment, the processing module 10001 is further configured to: If the test accuracy of the target detection model does not reach a preset accuracy threshold, the method further includes: selecting from the test data set each task scenario corresponding to test data whose test result does not match the real environment state; For each task scenario, the following are performed respectively: obtaining various types of new marine environment data and corresponding new detection results under the task scenario, and adding the task scenario, the various types of new marine environment data and the new detection result as a new sample data to the sample data set, and taking the real environment state described by the new detection result as a sample label of the new sample data; dividing a new training data set from the sample data set, and training the target detection model based on the new training data set to obtain a new target detection model.

[0107] In an optional embodiment, the parsing module 10002 can be used for: When the data fusion mode is decision-level fusion, obtaining a second fusion weight of each type of marine environment data under the decision-level fusion mode in the target task scene; Respectively parsing each type of marine environment data to obtain a corresponding sub-detection result; Respectively performing confidence assessment on each sub-detection result to obtain a corresponding confidence assessment result; Each sub-detection result whose confidence assessment result meets the preset assessment standard is taken as a positive sub-detection result, and its corresponding second fusion weight is maintained; Each sub-detection result whose confidence assessment result does not meet the preset assessment standard is taken as a negative sub-detection result, and its corresponding second fusion weight is adjusted to obtain an adjusted third fusion weight; Based on each positive sub-detection result and the corresponding second fusion weight, and in combination with each negative detection result and the corresponding third fusion weight, a sub-detection result fusion processing is performed to obtain a corresponding detection result.

[0108] Through the cooperation of the acquisition module, the processing module and the parsing module of the marine ecological environment detection device, low-correlation marine environment data can be screened out, multi-modal data can be processed by combining scene information and data attributes and adapting to the fusion mode, the sample training model can be updated, data can be supplemented for retraining when the model accuracy is not up to standard, and the weight of the sub-detection result can be adjusted for decision-level fusion, so as to fully exert the advantages of multi-modal data, improve the detection accuracy and model adaptability, and provide reliable support for marine ecological environment detection.

[0109] Based on the same inventive concept, the embodiments of the present application provide a computer device. The electronic device can realize the functions of the marine ecological environment detection device described above, and refer to Figure 11 The electronic device includes: At least one processor 11001 and a memory 11002 connected with the at least one processor 11001. In the embodiments of the present application, the specific connection medium between the processor 11001 and the memory 11002 is not limited, Figure 11 In the embodiments of the present application, the connection between the processor 11001 and the memory 11002 through the bus 11000 is taken as an example. The bus 11000 is represented by a thick line in Figure 11 The connection mode between other components is only schematically illustrated, and is not limited. The bus 11000 can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 11 In the embodiments of the present application, only one thick line is used to represent the bus 11000, but it does not mean that there is only one bus or only one type of bus. Alternatively, the processor 11001 can also be called a controller, and the name is not limited.

[0110] In the embodiments of the present application, the memory 11002 stores instructions executable by the at least one processor 11001, and the at least one processor 11001 can execute the method for detecting the marine ecological environment discussed above by executing the instructions stored in the memory 11002. The processor 11001 can realize the functions of each module of the apparatus shown in the figure. Among them, the processor 11001 is the control center of the apparatus, which can connect each part of the whole control device through various interfaces and lines, and realize the functions and process data of the apparatus by running or executing the instructions stored in the memory 11002 and calling the data stored in the memory 11002, thereby overall monitoring the apparatus. Figure 10

[0111] In a possible design, the processor 11001 can include one or more processing units, and the processor 11001 can integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface, application program and the like, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 11001. In some embodiments, the processor 11001 and the memory 11002 can be implemented on the same chip, and in some embodiments, they can also be implemented on separate chips respectively.

[0112] The processor 11001 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, which can realize or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method for detecting the marine ecological environment disclosed in the embodiments of the present application can be directly embodied as the execution of the hardware processor, or the execution of the combination of the hardware and software modules in the processor.

[0113] ​The memory 11002, as a non-volatile computer readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 11002 can include at least one type of storage medium, for example, can include flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. The memory 11002 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited thereto. The memory 11002 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used for storing program instructions and / or data.

[0114] By designing and programming the processor 11001, the code corresponding to the detection method of the marine ecological environment introduced in the foregoing embodiments can be fixed in the chip, so that the chip can execute the steps of the detection method of the marine ecological environment of the embodiments shown in the running. Figure 2 How to design and program the processor 11001 is a technology known to those skilled in the art, which will not be described here.

[0115] Based on the same inventive concept, the embodiments of the present application provide a computer readable storage medium storing a computer program executable by a computer device, which causes the computer device to execute the steps of the detection method of the marine ecological environment when the program runs on the computer device.

[0116] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0117] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0118] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0119] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 Figure 1 one or more flowcharts and / or blocks

[0120] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for detecting a marine ecological environment, characterized by, The method comprises: determining a target task scene, and obtaining scene description information of the target task scene; the scene description information is used for describing an actual marine environment under the target task scene; obtaining various types of marine environment data collected under the target task scene; different types of marine environment data have different perception modalities; based on the scene description information and attribute information of the various types of marine environment data, obtaining a comprehensive quality evaluation value of the actual marine environment, and selecting a data fusion mode matched with the comprehensive quality evaluation value; based on the data fusion mode, performing fusion processing on the various types of marine environment data to obtain fusion environment data with multiple modalities; analyzing the fusion environment data to obtain a detection result of the target task scene.

2. The method of claim 1, wherein, After obtaining the various types of marine environment data collected under the target task scene, and before obtaining the comprehensive quality evaluation value of the actual marine environment based on the scene description information and the attribute information of the various types of marine environment data, the method further comprises: for each type of marine environment data, respectively performing: from the marine environment data, deleting part of the data that does not meet a preset correlation degree condition between the target task scene, to obtain updated marine environment data.

3. The method of claim 1, wherein, The target task scene is to detect hydrological parameters of a marine environment, and the actual marine environment described by the scene description information is a hydrological comprehensive interference condition; based on the scene description information and the attribute information of the various types of marine environment data, obtaining a comprehensive quality evaluation value of the actual marine environment, and selecting a data fusion mode matched with the comprehensive quality evaluation value, comprises: for each type of marine environment data, respectively performing: taking the quality of the hydrological parameters represented by the hydrological related data contained in the marine environment data as the attribute information of the marine environment data; based on the hydrological comprehensive interference condition and the quality of the hydrological parameters, obtaining a corresponding hydrological detection evaluation value, and taking the hydrological detection evaluation value as the comprehensive quality evaluation value; based on the test accuracy of the hydrological parameters represented by the hydrological detection evaluation value, selecting an adaptive data fusion mode.

4. The method of claim 1, wherein, The target task scene is to perform target identification on marine organisms in a marine environment, and the actual marine environment described by the scene description information is a marine organism comprehensive survival condition; based on the scene description information and the attribute information of the various types of marine environment data, obtaining a comprehensive quality evaluation value of the actual marine environment, and selecting a data fusion mode matched with the comprehensive quality evaluation value, comprises: for each type of marine environment data, respectively performing: taking the target distinguishability represented by the marine organism related data contained in the marine environment data as the attribute information of the marine environment data; based on the marine organism comprehensive survival condition and the target distinguishability, obtaining a corresponding marine organism identification evaluation value, and taking the marine organism identification evaluation value as the comprehensive quality evaluation value; based on the identification accuracy of the marine organisms represented by the marine organism identification evaluation value, selecting an adaptive data fusion mode.

5. The method of claim 1, wherein, The target task scene is: abnormal early warning for marine biological density in a marine environment, and the actual marine environment described by the scene description information is: comprehensive survival of marine organisms. The comprehensive quality evaluation value of the actual marine environment is obtained based on the scene description information and the attribute information of each type of marine environment data, and a data fusion manner matched with the comprehensive quality evaluation value is selected, including: For each type of marine environment data, the marine organism density credibility represented by the marine organism related data contained in the marine environment data is taken as the attribute information of the marine environment data; Based on the comprehensive survival of marine organisms and the marine organism density credibility, a marine organism density early warning evaluation value is obtained, and the marine organism density early warning evaluation value is taken as the comprehensive quality evaluation value; Based on the early warning judgment accuracy represented by the marine organism density early warning evaluation value, an adaptive data fusion manner is selected.

6. The method according to any one of claims 1 to 5, wherein, The fusion environment data with multiple modalities is obtained by fusing the various types of marine environment data based on the data fusion manner, including: Obtaining first fusion weights of the various types of marine environment data under the data fusion manner under the target task scene; When the data fusion manner is data-level fusion, the various types of marine environment data are aligned in data format, and then fused by using the corresponding first fusion weights to obtain the fusion environment data with multiple modalities; When the data fusion manner is feature-level fusion, the environment features of the various types of marine environment data are respectively extracted, and then the obtained environment features are unified in dimension, and then fused by using the corresponding first fusion weights to obtain the fusion environment data with multiple modalities.

7. The method according to any one of claims 1 to 5, wherein The method further includes: When the data fusion manner is decision-level fusion, second fusion weights of the various types of marine environment data under the decision-level fusion manner under the target task scene are obtained; The various types of marine environment data are respectively analyzed to obtain corresponding sub-detection results; The confidence of each sub-detection result is evaluated to obtain a corresponding confidence evaluation result; Each sub-detection result whose confidence evaluation result meets a preset evaluation standard is taken as a positive sub-detection result, and its corresponding second fusion weight is kept; Each sub-detection result whose confidence evaluation result does not meet the preset evaluation standard is taken as a negative sub-detection result, and its corresponding second fusion weight is reduced and adjusted to obtain an adjusted third fusion weight; Based on the positive sub-detection results and the corresponding second fusion weights, and based on the negative sub-detection results and the corresponding third fusion weights, sub-detection result fusion processing is performed to obtain a corresponding detection result.

8. The method of any one of claims 1-5, wherein, The method is realized by a target detection model, and after the detection result of the target task scene is obtained, the method further includes: update the target task scene, the various types of marine environment data and the detection result as a new sample data to a sample data set, and take the real environment state described by the detection result as a sample label of the sample data; each other sample data in the sample data set comprises: an other task scene, various types of marine environment data collected under the other task scene and a corresponding other detection result; divide the sample data set into a training data set and a test data set; perform model training on the detection model to be trained based on the training data set, to obtain an initial detection model; test train the initial detection model by using the test data set, to obtain the target detection model.

9. The method of claim 8, wherein, If the test accuracy of the target detection model does not reach a preset accuracy threshold, the method further comprises: selecting, from the test data set, each task scene corresponding to test data whose test result does not match the real environment state; for each task scene, respectively performing: obtaining various types of new marine environment data under the task scene and a corresponding new detection result, and adding the task scene, the various types of new marine environment data and the new detection result as a new sample data to the sample data set, and taking the real environment state described by the new detection result as a sample label of the new sample data; divide a new training data set from the sample data set, and train the target detection model based on the new training data set, to obtain a new target detection model.

10. A device for detecting a marine ecological environment, characterized by comprising: The device comprises: an acquisition module: configured to determine a target task scene, and acquire scene description information of the target task scene; the scene description information is used to describe an actual marine environment under the target task scene, and various types of marine environment data collected under the target task scene are acquired; different types of marine environment data have different perception modalities; a processing module: configured to obtain a comprehensive quality evaluation value of the actual marine environment based on the scene description information and attribute information of the various types of marine environment data, and select a data fusion mode matched with the comprehensive quality evaluation value; and perform fusion processing on the various types of marine environment data based on the data fusion mode, to obtain fusion environment data with multiple modalities; an analysis module: configured to analyze the fusion environment data, to obtain a detection result of the target task scene.

11. A computer device, comprising: comprise: a memory: configured to store a computer program; a processor: configured to execute the computer program stored in the memory, to implement the method steps in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer program stored in the memory can be executed by the computer device, and when the computer program runs on the computer device, the computer device is caused to execute the steps of the method in any one of claims 1-9.

Images