Sonar intelligent interpretation method and system

Abstract

The invention provides a sonar intelligent interpretation method and system, and the method comprises the steps: integrating end-side equipment (disposed on a ship-borne platform) and side-side equipment (disposed in a shore-based machine room) to form a sonar intelligent interpretation system; through the intelligent sonar interpretation system, full-process automatic processing of sonar interpretation from data acquisition of sonar images to interpretation is realized, the interpretation efficiency of the sonar images is effectively improved, and in addition, the intelligent sonar interpretation system also can be used for realizing automatic interpretation of the sonar images on the basis of mass sonar image data (namely sonar images stored in a data warehouse) obtained through actual measurement. Model training of the sonar interpretation model and training examination of interpretation personnel are synchronously achieved, a sustainable closed-loop optimization system is formed, and the problems that in traditional sonar interpretation, the interpretation accuracy is low, the data reuse rate is poor, and personnel training examination is difficult are solved.

Description

Technical Field

[0001] This application relates to the field of sonar technology, and more specifically, to a sonar intelligent interpretation method and system. Background Technology

[0002] Sonar technology is an acoustic method that utilizes the underwater propagation characteristics of sound waves for detection and imaging. It has been widely applied in fields such as marine mapping, underwater security, and military reconnaissance. Among them, imaging sonar is a key branch of sonar technology, including forward-looking sonar, side-scan sonar, and multibeam sonar. Forward-looking sonar is similar to radar, capable of scanning the water ahead in real time and providing dynamic images; side-scan sonar emits sound waves from the side of the ship's hull to generate high-resolution seabed topographic maps, often used for detecting underwater pipelines or shipwrecks; multibeam sonar uses multiple sound beams to scan simultaneously, constructing a three-dimensional underwater model, suitable for large-area mapping.

[0003] Sonar images generated by the aforementioned imaging sonar are typically presented in grayscale or pseudo-color. These images contain highlight areas, shadow areas, and background noise, which can help identify underwater targets (such as schools of fish, reefs, shipwrecks, etc.).

[0004] However, traditional sonar detection currently relies on manual interpretation. Operators need to extract sonar images from raw echo data and analyze the detected targets frame by frame. While this method is reliable, it is limited by human visual fatigue and differences in experience, resulting in low efficiency in interpreting sonar images. Summary of the Invention

[0005] In view of this, this application provides a sonar intelligent interpretation method and system. By integrating end-side equipment (deployed on a shipboard platform) and side-side equipment (deployed in a shore-based equipment room) to form a sonar intelligent interpretation system, the sonar intelligent interpretation system realizes the fully automated processing of sonar image data acquisition and interpretation, effectively improving the interpretation efficiency of sonar images. In addition, the sonar intelligent interpretation system can also simultaneously realize the model training of sonar interpretation models and the training and assessment of interpreters based on the massive sonar image data obtained from actual measurements (i.e., sonar images stored in a data warehouse), forming a sustainable closed-loop optimization system. This is conducive to solving many problems in traditional sonar interpretation, such as low interpretation accuracy, poor data reuse rate, and difficulty in personnel training and assessment.

[0006] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings.

[0007] In a first aspect, embodiments of this application provide a sonar intelligent interpretation method applied to a sonar intelligent interpretation system. The sonar intelligent interpretation system includes: an end-side device and a side-side device, wherein the end-side device is deployed on a shipboard platform, and the side-side device is deployed in a shore-based engine room. A communication connection is established between the side-side device and the end-side device. The sonar intelligent interpretation method includes: The end-side device classifies and stores the acquired raw sonar images into a data warehouse according to the data acquisition scenario corresponding to the sonar images; According to the model training objective of the sonar interpretation model, the side device obtains sonar images that match the model training objective from the data warehouse as the model training dataset of the sonar interpretation model, and trains the sonar interpretation model using the model training dataset to obtain the trained target sonar interpretation model. The side-side device performs model evaluation and verification on the target sonar interpretation model, and sends the target sonar interpretation model that has passed the model evaluation and verification to the end-side device; The end-side device receives and deploys the target sonar interpretation model, inputs the acquired original sonar image into the target sonar interpretation model, and obtains the first sonar interpretation result output by the target sonar interpretation model for the original sonar image.

[0008] Secondly, embodiments of this application provide a sonar intelligent interpretation system, which includes: an end-side device and a side-side device, wherein the end-side device is deployed on a shipboard platform, and the side-side device is deployed in a shore-based engine room, and a communication connection is established between the side-side device and the end-side device. The end-side device is used to classify and store the acquired raw sonar images into a data warehouse according to the data acquisition scenario corresponding to the sonar images; The side device is used to obtain sonar images matching the model training purpose from the data warehouse according to the model training purpose of the sonar interpretation model, as the model training dataset of the sonar interpretation model, and to train the sonar interpretation model using the model training dataset to obtain the trained target sonar interpretation model. The side-side device is used to evaluate and verify the target sonar interpretation model, and to send the target sonar interpretation model that has passed the evaluation and verification to the end-side device. The end-side device is used to receive and deploy the target sonar interpretation model, input the acquired original sonar image into the target sonar interpretation model, and obtain the first sonar interpretation result output by the target sonar interpretation model for the original sonar image.

[0009] The technical solutions provided by the embodiments of this application may include the following beneficial effects: This application provides a sonar intelligent interpretation method and system. The system integrates end-side equipment (deployed on a shipboard platform) and side-side equipment (deployed in a shore-based equipment room) to form a sonar intelligent interpretation system. This system automates the entire sonar interpretation process, from data acquisition to interpretation of sonar images, effectively improving the efficiency of sonar image interpretation. Furthermore, based on massive amounts of sonar image data obtained from actual measurements (i.e., sonar images stored in a data warehouse), the system can simultaneously train sonar interpretation models and conduct training and assessment of interpreters, forming a sustainable closed-loop optimization system. This helps solve many problems in traditional sonar interpretation, such as low interpretation accuracy, poor data reuse, and difficulties in personnel training and assessment. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This paper shows a schematic diagram of the structure of a sonar intelligent interpretation system provided in an embodiment of this application; Figure 2 A flowchart illustrating a sonar intelligent interpretation method provided in an embodiment of this application is shown; Figure 3 This illustration shows a schematic diagram of the device structure of an end-side device according to an embodiment of this application; Figure 4 A schematic diagram of the device structure of a side device provided in an embodiment of this application is shown. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0013] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0014] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0015] Here, the sonar intelligent interpretation method provided in this application embodiment can be applied to a sonar intelligent interpretation system. Figure 1 This paper illustrates a schematic diagram of the structure of a sonar intelligent interpretation system provided in an embodiment of this application, as shown below. Figure 1 As shown, the sonar intelligent interpretation system includes: end-side equipment and side-side equipment. The end-side equipment is deployed on a shipboard platform (i.e., an equipment installation and operation platform deployed on a ship, used to carry various functional systems and perform specific tasks, such as the sonar image acquisition and interpretation tasks involved in this application). The side-side equipment is deployed in a shore-based machine room (i.e., a machine room facility deployed on the land side). A communication connection is established between the side-side equipment and the end-side equipment, enabling remote communication and data transmission.

[0016] It should be noted that multiple end-side devices can be deployed on a shipboard platform, and multiple side-side devices can also be deployed in a shore-based machine room. This application does not limit the specific number of end-side devices or side-side devices included in the sonar intelligent interpretation system.

[0017] In the sonar intelligent interpretation system provided in this application embodiment, the end-side device is mainly responsible for low-latency response and local decision-making, while the edge-side device is mainly responsible for large-scale model training, long-term model optimization, and sonar interpreter management (e.g., regularly conducting online training and assessments for sonar interpreters). Thus, in the sonar intelligent interpretation system, a continuous evolutionary closed loop of "data-driven - model iteration - deployment verification" is formed through reliable end-edge data, model, and operation and maintenance interaction.

[0018] Specifically, in a sonar intelligent interpretation system, the interaction process between the end-side device and the edge-side device can be briefly summarized as follows: a. Secure and reliable communication: End-side devices and edge-side devices exchange data and models through an encrypted channel (authentication + encryption). That is, the end-side device transmits the acquired sonar images and the sonar interpretation results of the sonar images to the edge-side device, and the edge-side device transmits the sonar interpretation model used for sonar interpretation to the end-side device. Downlink model packets support verification and signature. When the network fluctuates, the end-side device can use caching and batch uploading strategies to send relevant data to the edge-side device.

[0019] b. Data Upload: The edge device uploads key snapshots of sonar images, low-confidence / abnormal sonar image samples, and operation logs to the side device according to rules. The side device performs quality checks on the received sonar image samples and includes qualified samples into the training pool of the sonar interpretation model. In addition, for difficult samples uploaded by the edge device (i.e., sonar image samples with high sonar interpretation difficulty), a manual annotation process can be triggered on the side device (mainly from the annotators or industry experts connected to the personnel training and assessment subsystem on the side device) to obtain the manual interpretation results of the difficult samples as model training data.

[0020] c. Model Download: After completing training and evaluation, the edge devices can download the candidate sonar interpretation model to the end devices. The download process supports canary release (first downloading the sonar interpretation model to a small number of end devices to observe its online performance). If the online performance indicators of the sonar interpretation model meet the expectations, the sonar interpretation model can be downloaded to all end devices. If an anomaly occurs, fast rollback (i.e., rollback to the previous version of the sonar interpretation model) is supported.

[0021] d. Online monitoring and reporting: The edge device reports the performance indicators (detection rate, confidence distribution, number of alarms) of the sonar interpretation model to the side device in real time. Based on this telemetry data, the side device can determine whether it is necessary to trigger retraining or threshold adjustment of the current version of the sonar interpretation model.

[0022] e. Human-machine closed loop: Operators of edge devices can manually confirm and submit annotations for low-confidence or complex scenarios. These annotations will be sent by the edge device to the data warehouse on the side device for storage, and will be given priority in the new model training data of subsequent versions of sonar interpretation models. This will help drive the sonar interpretation model to continuously improve its sonar interpretation capabilities through human experience.

[0023] f. Version and Audit: Side devices will record every model version, model training dataset change, and model distribution history to facilitate subsequent backtracking, compliance, and accountability.

[0024] To facilitate understanding of the embodiments of this application, the following description is provided in conjunction with... Figure 1 The sonar intelligent interpretation system shown herein provides a detailed description of a sonar intelligent interpretation method provided in the embodiments of this application.

[0025] Reference Figure 2 As shown, Figure 2 The diagram illustrates a flowchart of a sonar intelligent interpretation method provided in an embodiment of this application, wherein the sonar intelligent interpretation method can be applied to... Figure 1 In the sonar intelligent interpretation system shown, the sonar intelligent interpretation method includes steps S201-S204; specifically: S201, the end-side device classifies and stores the acquired raw sonar images into a data warehouse according to the data acquisition scenario corresponding to the sonar images.

[0026] Here, referring to the aforementioned description of the sonar intelligent interpretation system, it can be seen that in the sonar intelligent interpretation system, the edge device is used to acquire sonar images that need to be interpreted in real time, and automatically interprets the acquired sonar images based on its own deployed sonar interpretation model (i.e., the sonar interpretation model issued by the edge device).

[0027] In the embodiments of this application, such as Figure 1 As shown, the sonar intelligent interpretation system can be further divided into an interpretation intelligent agent subsystem and a data analysis subsystem deployed on the end device. On the end device side, the data analysis subsystem mainly performs steps such as data acquisition and preliminary analysis processing of sonar images. That is, on the end device side, the data analysis subsystem mainly performs the above steps S201.

[0028] Specifically, on the end-side device, regarding the sonar image data acquisition section, the data analysis subsystem can execute the above step S201 according to the method shown in steps a1-a3 below: Step a1: In response to the triggering of preset conditions, the original sonar image is automatically acquired, and the environmental parameters related to the original sonar image are recorded simultaneously.

[0029] Here, during the continuous reception of sonar echo signals by the end-side equipment, the data analysis subsystem will automatically determine whether it is necessary to capture sonar images (i.e., whether it is necessary to collect raw sonar images) and generate snapshots based on preset conditions (such as the signal-to-noise ratio of the sonar echo signal being lower than a preset signal-to-noise ratio threshold, abnormal motion trajectory during sonar detection, or detection targets with prominent echo intensity). The captured sonar images (i.e., the raw sonar images mentioned above) not only retain the raw sonar data (ping signal) but also record the preprocessing results to ensure data traceability.

[0030] Specifically, in practical applications, the data analysis subsystem can typically complete the triggering judgment of the aforementioned preset conditions and save a snapshot within milliseconds, obtaining the sonar image acquired when the aforementioned preset conditions are triggered (i.e., the aforementioned original sonar image). Among them, when the end-side device detects strong interference signals in turbid waters, the data analysis subsystem will immediately trigger the snapshot function and simultaneously record relevant environmental parameters to avoid missing key information that is helpful for sonar interpretation. Thus, through this dual saving mechanism of "original sonar image data + snapshot", the data analysis subsystem can form a complete data chain, which is not only convenient for subsequent review and analysis, but also ensures reliability in complex sonar detection scenarios.

[0031] It should be noted that the above-mentioned environmental parameters include, but are not limited to: timestamp parameters, platform attitude parameters of the shipborne platform, and underwater environmental parameters at the sonar image acquisition location; the specific types and quantities of the above-mentioned environmental parameters are not limited in this application embodiment.

[0032] In addition, it should be noted that the data analysis subsystem is also equipped with an anomaly alarm function, which enables the data analysis subsystem to simultaneously push a prompt to the operator on the end device side when it detects a suspicious sonar detection target (e.g., displaying relevant prompts on the corresponding display screen of the end device), so that the operator can intervene quickly and improve the safety and timeliness of the overall operation.

[0033] Specifically, after acquiring the raw sonar image, as an optional embodiment, the data analysis subsystem can also perform preliminary preprocessing on the raw sonar image. The specific preprocessing methods may include, but are not limited to: noise reduction processing, filtering processing, geometric correction processing, and extracting candidate regions of interest from the raw sonar image.

[0034] It should be noted that during the above preprocessing process, the data analysis subsystem can automatically identify geometric deviations caused by attitude changes of the sonar detection equipment on the shipboard platform, and correct these geometric deviations in the background to ensure the consistency and accuracy of the original sonar image data.

[0035] It should be noted that the data analysis subsystem also supports human-machine collaborative data governance. On the edge device side, operators can quickly review the candidate regions of interest identified in the original sonar images through a visual dashboard. For example, they can make one-click corrections to candidate targets with low confidence. All correction operations are recorded in real time and sent back to the edge device for subsequent optimization of the sonar interpretation model.

[0036] In addition, as another optional embodiment, the data analysis subsystem can also integrate target trajectory playback and hot spot magnification functions, enabling operators to quickly locate and trace anomalies in specific areas, thereby improving the interpretability and analysis efficiency of the original sonar images, which is conducive to improving the data quality of sonar images and reducing misjudgments of sonar detection targets due to noise or attitude interference.

[0037] Step a2: Based on the environmental parameters, determine the target data acquisition scene that matches the environmental parameters from multiple data acquisition scenes corresponding to the sonar image.

[0038] Here, the multiple data acquisition scenarios corresponding to sonar images refer to multiple unique data acquisition scenarios that may be encountered during sonar image acquisition; for example, multiple data acquisition scenarios may include, but are not limited to: high-noise water areas, weak echo areas, multipath interference scenarios, bubble interference water areas, etc.

[0039] Specifically, as an optional embodiment, the data analysis subsystem can obtain the underwater environmental parameters at the sonar image acquisition location from the above-mentioned environmental parameters, thereby determining the data acquisition scenario that matches the underwater environmental parameters from the above-mentioned multiple data acquisition scenarios as the target data acquisition scenario.

[0040] It should be noted that, based on the above preprocessing, when multiple regions of interest are identified in the original sonar image through the above preprocessing, the data analysis subsystem can also perform deep classification and identification of the identified regions of interest.

[0041] Step a3: Store the original sonar image in the sonar image dataset corresponding to the target data acquisition scene in the data warehouse.

[0042] Here, on the edge device side, the data analysis subsystem can convert the raw sonar images into structured formats (such as COCO format, JSON format, etc.) and send the raw sonar images and their scene classification results (i.e. the target data acquisition scene mentioned above) to the edge device. The edge device can store the received raw sonar images into the sonar image dataset corresponding to the target data acquisition scene in the data warehouse (that is, the data warehouse is located on the edge device side), forming a unified data asset.

[0043] Regarding the sonar image datasets stored in the aforementioned data warehouse, which belong to various data acquisition scenarios, it should be noted that the purpose of the sonar image data stored in the data warehouse is multi-dimensional, for example, such as... Figure 1 As shown, the sonar intelligent interpretation system can be further divided into a model training and continuous optimization subsystem and a personnel training and assessment subsystem deployed on the edge device. On the edge device side, the model training and continuous optimization subsystem can obtain sonar images from the data warehouse as model training data required for sonar interpretation model training; on the other hand, the personnel training and assessment subsystem can also obtain sonar images from the data warehouse as examination materials required for the training and assessment of sonar interpretation personnel.

[0044] In this embodiment, based on the data acquisition and data storage mechanism shown in steps a1-a3 above, the sonar intelligent interpretation system can rapidly expand the data warehouse and dynamically update the data warehouse as sonar images are continuously acquired, avoiding the limitations of traditional static databases in new scenarios. Furthermore, the sonar images stored in the data warehouse have multiple uses in the sonar intelligent interpretation system (they can be used as model training data or as personnel training and assessment materials), effectively improving the data reuse rate of sonar images and fundamentally solving the problem of poor data reuse rate of sonar image data in traditional sonar interpretation.

[0045] S202, the side device, according to the model training objective of the sonar interpretation model, obtains sonar images matching the model training objective from the data warehouse as the model training dataset of the sonar interpretation model, and trains the sonar interpretation model using the model training dataset to obtain the trained target sonar interpretation model.

[0046] In the embodiments of this application, such as Figure 1As shown, the sonar intelligent interpretation system can be further divided into a model training and continuous optimization subsystem and a personnel training and assessment subsystem deployed on the side device. On the side device, the model training and continuous optimization subsystem mainly performs steps such as model training, verification, and testing of the sonar interpretation model. That is, on the side device, the model training and continuous optimization subsystem mainly performs the above steps S202.

[0047] Here, the model training and continuous optimization subsystem can read sonar images that match the above-mentioned model training objectives from the data warehouse, based on the model training objectives (including but not limited to: sonar detection scenarios such as strong noise, bubble interference waters, or sonar detection tasks such as surveying, security, and reconnaissance), and divide the read sonar images into a model training dataset (for model training) and a model testing dataset (for model evaluation and validation).

[0048] Specifically, as an optional embodiment, the model training and continuous optimization subsystem can automatically select the target image detection model with the highest matching degree to the above-mentioned model training objective from a variety of image detection models with different structures as the original model of the sonar interpretation model. The sonar images in the above-mentioned model training dataset are input into the target image detection model (i.e., the sonar interpretation model), and the target detection model outputs the target detection result for each sonar image (i.e., the detection target included in the sonar image and the category to which the detection target belongs). Based on the loss between the target detection result of each sonar image and the real detection result (i.e., the manually labeled result), the target image detection model is trained until the target image detection model converges, and the target sonar interpretation model (i.e., the converged target image detection model) is obtained.

[0049] It should be noted that, considering that the amount of sonar image data in different data acquisition scenarios in the data warehouse may vary (for example, the number of effective sonar images acquired in more complex data acquisition scenarios such as strong noise and bubble-interference waters may be smaller), when the model training objective is matched with the aforementioned data acquisition scenario with a smaller amount of data, there may be a situation where the acquired model training data is insufficient. In this case, the model training and continuous optimization subsystem can use simulation applications such as MATLAB to generate simulated sonar images that match the model training objective to make up for the data volume gap in the model training data, so as to ensure that the trained target sonar interpretation model can have the ability to cope with complex environments.

[0050] S203, the side device performs model evaluation and verification on the target sonar interpretation model, and sends the target sonar interpretation model that has passed the model evaluation and verification to the end device.

[0051] Here, as an optional embodiment, before evaluating and validating the target sonar interpretation model, the model training and continuous optimization subsystem can also perform lightweight processing on the target sonar interpretation model to reduce the number of model parameters, making it run faster and more energy-efficiently on edge devices with limited computing power.

[0052] It should be noted that the specific processing methods for the above-mentioned lightweighting process include, but are not limited to, model distillation, pruning and quantization, etc., and the embodiments of this application do not limit them in any way.

[0053] Specifically, during step S203, on the side device side, the model training and continuous optimization subsystem can evaluate and validate the target sonar interpretation model according to the methods shown in steps b1-b4 below: Step b1: Obtain sonar images that match the model training objective from the data warehouse as the model test dataset.

[0054] Here, referring to the content of step S202 above, it can be seen that the model training and continuous optimization subsystem can read sonar images that match the above-mentioned model training purpose from the data warehouse, and divide the read sonar images into a model training dataset (for model training) and a model test dataset (for model evaluation and verification). That is, the sonar images used as the model test dataset can be the same as the sonar images used as the model training dataset, or they can be different from the sonar images used as the model training dataset. This application embodiment does not impose any mandatory limitations on this.

[0055] Step b2: Input the model test dataset into the target sonar interpretation model to obtain the second sonar interpretation result output by the target sonar interpretation model for the model test dataset.

[0056] Specifically, similar to the model training part in step S202 above, the model training and continuous optimization subsystem inputs the sonar images in the above model test dataset into the target sonar interpretation model. The target sonar interpretation model can output the second sonar interpretation result of the target sonar interpretation model for each sonar image by performing target identification and detection on the detection targets contained in each input sonar image (i.e., the detection targets included in the sonar image, the category to which the detection targets belong, the confidence level of the detection targets belonging to the category, etc.).

[0057] Step b3: Based on the second sonar interpretation result and the model test data, calculate the index values ​​corresponding to multiple quantitative indicators used to evaluate the model performance of the target sonar interpretation model, and determine the model evaluation result of the target sonar interpretation model based on the index values ​​corresponding to the multiple quantitative indicators.

[0058] Here, the aforementioned quantitative metrics may include, but are not limited to: mAP (mean precision), Precision, Recall, and F1 score.

[0059] Specifically, the values ​​of the aforementioned quantitative indicators are used to reflect the model performance of the target sonar interpretation model. Based on this, as an optional embodiment, for each of the quantitative indicators, it can be determined whether the value of the quantitative indicator reaches the corresponding indicator threshold (for example, whether the value of the accuracy indicator reaches a preset accuracy threshold). If the values ​​of the multiple quantitative indicators all reach their respective indicator thresholds, it can be determined that the target sonar interpretation model has passed the model evaluation; otherwise, it can be determined that the target sonar interpretation model has failed the model evaluation.

[0060] Step b4: The side device performs scenario-based verification of the target sonar interpretation model based on the simulation data and measured data corresponding to the sonar image in the preset data acquisition scenario, and obtains the model verification result of the target sonar interpretation model.

[0061] Here, the preset data acquisition scenario refers to the complex data acquisition scenario unique to sonar, such as complex data acquisition scenarios where it is difficult to acquire sonar images, such as in high-noise waters, weak echo areas, and multipath interference scenarios.

[0062] Here, the simulation data refers to simulated sonar images generated for the aforementioned preset data acquisition scenario (e.g., in simulation applications such as MATLAB, scenario parameters that conform to the aforementioned preset data acquisition scenario can be configured to obtain a simulated data acquisition scenario that matches the aforementioned preset data acquisition scenario, thereby simulating the acquisition of sonar image data under the simulated data acquisition scenario to obtain the aforementioned simulation data), while the measured data refers to sonar images obtained from the data warehouse that match the preset data acquisition scenario (e.g., referring to the relevant description of the original sonar images in the aforementioned steps, it can be seen that the sonar images stored in the data warehouse are all real sonar images actually acquired by the end-side device in the waters where the shipborne platform is located).

[0063] Specifically, in the scenario-based verification phase, the model training and continuous optimization subsystem will test the target sonar interpretation model in the complex environment unique to sonar (i.e., the aforementioned preset data acquisition scenario). (For example, in high-noise waters, weak echo areas, and multipath interference scenarios, the robustness of the target sonar interpretation model is comprehensively verified by injecting simulation data and measured data.) The scenario-based verification may include, but is not limited to, the recognition accuracy, latency performance, and resource consumption exhibited by the target sonar interpretation model during the aforementioned scenario-based verification process, thereby ensuring that the target sonar interpretation model that passes the scenario-based verification can run stably on the edge device.

[0064] Here, when multiple end-side devices are deployed on the shipboard platform, the sonar intelligent interpretation system can distribute the target sonar interpretation model, which has been evaluated and verified by the model, to the end-side devices using the following method: On one side of the edge device, the model training and continuous optimization subsystem, in response to the target sonar interpretation model passing the model evaluation and verification, distributes the target sonar interpretation model to a preset number of edge devices.

[0065] Here, the preset number is less than the number of devices of the multiple end-side devices. That is, on the side-side device side, the model training and continuous optimization subsystem can distribute the target sonar interpretation model to some end-side devices (i.e., the preset number of end-side devices) in response to the target sonar interpretation model passing the model evaluation and verification.

[0066] On the end-side device (i.e., the aforementioned preset number of end-side devices) that receives the target sonar interpretation model, in response to receiving the target sonar interpretation model, the end-side device can replace the deployed old version of the sonar interpretation model (equivalent to the sonar interpretation model issued by the side-side device at a historical moment, and also equivalent to the sonar interpretation model deployed by the end-side device before receiving the target sonar interpretation model) with the target sonar interpretation model, and monitor the online performance of the target sonar interpretation model in real time, and determine the online performance result of the target sonar interpretation model (equivalent to real-time monitoring of the sonar interpretation model's sonar interpretation accuracy, false alarm rate, and other evaluation indicators used to evaluate its online performance, thereby quantitatively representing the online performance result of the target sonar interpretation model based on the index value of the evaluation indicators).

[0067] Building upon this, a pre-configured online performance standard (e.g., better performance than the older version of the sonar interpretation model) can be set on the edge device side to measure whether the target sonar interpretation model meets the expected online performance standard. This allows the edge device to promptly retrieve valid sonar image data from the edge device side as training data for model fine-tuning when the target sonar interpretation model fails to meet this standard. This fine-tuning (equivalent to model optimization) of the target sonar interpretation model yields a new version with superior performance. Specifically: On the side of the receiving end-side device (i.e., the aforementioned preset number of end-side devices) that receives the target sonar interpretation model, if the end-side device responds to the online performance result not meeting the preset online performance standard of the model, it can automatically switch the target sonar interpretation model to the older version of the sonar interpretation model.

[0068] The edge device (i.e., the aforementioned preset number of edge devices) can obtain the sonar image in which the sonar interpretation result of the target sonar interpretation model is incorrect from the online performance data of the target sonar interpretation model as a second misjudgment sample, and send the second misjudgment sample to the side device (mainly the model training and continuous optimization subsystem on one side of the side device).

[0069] On the side device side, in response to receiving the second misjudged sample, the model training and continuous optimization subsystem can use the second misjudged sample as the model fine-tuning training dataset, and fine-tune the target sonar interpretation model according to the model fine-tuning training dataset to obtain the fine-tuned target sonar interpretation model as the new version of the sonar interpretation model (wherein, the specific training method of fine-tuning training is similar to the model training method in the aforementioned step S202, and the repetition will not be repeated here).

[0070] The model training and continuous optimization subsystem can distribute the new sonar interpretation model to edge devices (i.e., the aforementioned preset number of edge devices) so that the edge devices can continue to monitor the online performance of the new sonar interpretation model in real time. The above steps are repeated until the online performance of the sonar interpretation model deployed on the edge devices can reach the aforementioned online performance standard.

[0071] S204, the end-side device receives and deploys the target sonar interpretation model, inputs the acquired original sonar image into the target sonar interpretation model, and obtains the first sonar interpretation result output by the target sonar interpretation model for the original sonar image.

[0072] Here, as Figure 1 As shown, on the edge device side, the data analysis subsystem is mainly used to collect sonar image data, while the interpretation intelligent agent subsystem, which is also deployed on the edge device, is used to realize target detection, recognition, localization and alarm of sonar images on the edge. That is, on the edge device side, the target sonar interpretation model is deployed in the interpretation intelligent agent subsystem, so that the interpretation intelligent agent subsystem can perform sonar interpretation on the sonar images (such as the original sonar images mentioned above) collected in real time by the data analysis subsystem according to the deployed target sonar interpretation model.

[0073] Specifically, taking the original sonar image mentioned above as an example, the interpreting intelligent agent subsystem on the end-side device can interpret the original sonar image according to the following steps c1-c2: Step c1: Preprocess the original sonar image to obtain the preprocessed original sonar image as the sonar image to be detected.

[0074] Here, the interpretation agent subsystem can obtain raw sonar images from the VGA video stream acquired by the data analysis subsystem; when preprocessing the raw sonar images, the specific preprocessing methods adopted by the interpretation agent subsystem may include, but are not limited to: noise reduction, cropping (e.g., cropping only the region of interest), normalization, etc.

[0075] Step c2: Input the sonar image to be detected into the target sonar interpretation model, perform target detection and recognition on the sonar image to be detected through the target sonar interpretation model, and output the target detection and recognition result as the first sonar interpretation result.

[0076] Here, the target detection and recognition results include: the image position (i.e., pixel position) of the detected target in the sonar image to be detected, the target type to which the detected target belongs, and the confidence level of the detected target belonging to the target type.

[0077] Here, in addition to the target detection and recognition results mentioned above, on the end-side device side, the interpreting intelligent agent subsystem can also determine the true geographical location (i.e., latitude and longitude) of the target by using the position of the shipborne platform and the relative position between the target and the shipborne platform, according to the method shown in steps d1-d3 below. The image detection result of the target (i.e., the target detection and recognition result mentioned above) and the true geographical location are used together as the first sonar interpretation result corresponding to the sonar image to be detected. Specifically: Step d1: Obtain the BeiDou positioning information of the shipborne platform through the BeiDou module installed on the shipborne platform.

[0078] here, Figure 3 This application provides a schematic diagram of the device structure of an end-side device according to an embodiment of the present application. Figure 3 As shown, apart from the data analysis subsystem and the interpretation intelligent agent subsystem deployed inside the edge device, from the perspective of the device structure visible from the outside, the edge device may include an interpretation device for performing sonar interpretation and a display for visualizing the sonar interpretation results to the operator; wherein, the display is connected to the main body of the edge device through a display linkage, and the operator can adjust the position of the display relative to the main body of the device by adjusting the display linkage (for example, when the display linkage is in the retracted state, the display can be close to the side of the main body of the device; when the operator needs to view the display, the display linkage can be adjusted to the open position so that the display can be adjusted to face the operator).

[0079] Specifically, such as Figure 3As shown, the end-side device can obtain the BeiDou positioning information of the shipborne platform through the BeiDou external antenna (i.e., BeiDou module) installed on the shipborne platform.

[0080] Step d2: Extract multiple key parameters from the sonar image to be detected using an optical character recognition model.

[0081] Here, the sonar image to be detected is input into the OCR (Optical Character Recognition) model. The OCR model can extract parameters such as distance range and hull angle from the sonar image to be detected as the above-mentioned key parameters (equivalent to the multiple key auxiliary parameters being parameters about the relative position between the target and the shipborne platform).

[0082] Step d3: Obtain the image location of the detected target from the target detection and recognition results, and fuse the image location with the BeiDou positioning information and the multiple key parameters to obtain the latitude and longitude location and relative orientation information of the detected target as the first sonar interpretation result.

[0083] Here, the relative orientation information refers to the relative orientation information between the detection target and the shipborne platform. That is, through the above-mentioned fusion calculation, the intelligent body subsystem can also obtain the real geographical location (i.e., latitude and longitude) of the detection target and the relative orientation between the detection target and the shipborne platform, thereby enabling more accurate positioning and tracking of the detection target.

[0084] Building upon this, on the end-side device side, the intelligent agent interpretation subsystem can also maintain a historical database composed of recently acquired historical sonar images by periodically performing sonar image detection on the waters where the shipborne platform is located, as shown in steps e1-e5 below. This allows it to compare the target state corresponding to the detected target in the currently acquired sonar image using time-series consistency. Specifically: Step e1: Input the multiple frames of historical sonar images continuously acquired in the previous time period into the target sonar interpretation model to obtain the historical sonar interpretation results output by the target sonar interpretation model for the multiple frames of historical sonar images.

[0085] Here, the historical sonar interpretation results include: multiple historical detection targets identified from the multiple frames of historical sonar images. The specific implementation of step e1 can be referred to the specific implementation of steps c1-c2 above. The repeated parts will not be repeated here.

[0086] Step e2: Input the latest sonar image acquired within the current time period into the target sonar interpretation model to obtain the latest sonar interpretation result output by the target sonar interpretation model for the latest sonar image.

[0087] Here, the latest sonar interpretation results include the latest detection target identified from the latest sonar image. The specific implementation of step e2 can also refer to the specific implementation of steps c1-c2 above, and the repeated parts will not be repeated here.

[0088] Step e3: Based on the latest sonar interpretation results and the historical sonar interpretation results, determine whether the number of times the latest detected target appears in the multi-frame historical sonar images is greater than or equal to a preset number threshold.

[0089] Here, based on the latest sonar interpretation results, the interpretation agent subsystem can determine the latest detection target. After the latest detection target is known, the interpretation agent subsystem can retrieve the number of historical detection targets that are the same as the latest detection target from the multiple historical detection targets included in the historical sonar interpretation results, and then obtain the number of times the latest detection target appears in the multiple frames of historical sonar images.

[0090] It should be noted that the preset number of times threshold can be 3 times or 4 times. This application embodiment does not limit the specific value of the preset number of times threshold.

[0091] Step e4: If the end-side device determines that the number of occurrences is greater than or equal to the preset number threshold, then in the latest sonar interpretation result, the latest detected target is marked as the real detected target; For example, taking a preset threshold of 3 times as an example, if the latest detected target appears more than or equal to 3 times in multiple frames of historical sonar images, the latest detected target can be marked with a red box in the latest sonar image, so that the operator can determine that the latest detected target marked with a red box belongs to the real detected target that appears multiple times in a row based on the red box mark carried by the latest detected target.

[0092] Step e5: If the end-side device determines that the number of occurrences is less than the preset number threshold, then in the latest sonar interpretation result, the latest detected target is marked as a suspected detected target.

[0093] For example, taking a preset threshold of 3 times as an example, if the latest detected target appears less than 3 times in multiple frames of historical sonar images, the latest detected target can be marked with a yellow box (i.e., a mark that is different from the mark corresponding to the above-mentioned real detected target) in the latest sonar image. This allows the operator to determine, based on the yellow box mark carried by the latest detected target, that the latest detected target marked with a yellow box belongs to a suspected detected target that appears occasionally. Thus, through the judgment mechanism shown in steps e1-e5 above, abnormal alarms caused by single-frame sonar image interpretation errors are reduced.

[0094] In this application embodiment, referring to the foregoing description of the sonar intelligent interpretation system, it can be seen that, in addition to achieving fully automated processing of the entire process from sonar image data acquisition to interpretation based on the methods shown in the above steps, as an optional embodiment, on the side-side device, a personnel training and assessment subsystem deployed on the side-side device can also be used to train and assess interpretation personnel according to the methods shown in steps f1-f4 below. Specifically: Step f1: Based on the training and assessment needs of sonar interpreters, retrieve sonar images from the data warehouse that match the training and assessment needs as training and assessment materials.

[0095] here, Figure 4 This application provides a schematic diagram of the device structure of a side-mounted device according to an embodiment of the present application. Figure 4 As shown, apart from the personnel training and assessment subsystem and the model training and continuous optimization subsystem deployed inside the edge device, from the perspective of the visible device structure, the end device can include a storage server for data storage and an intelligent training interactive all-in-one machine for supporting interaction with the client corresponding to the sonar interpreter.

[0096] Specifically, the purpose of the personnel training and assessment subsystem is to establish a standardized and quantifiable training and assessment closed loop, enabling sonar interpreters to master system operation and target interpretation in a short period of time, reducing subjective bias, and transforming test questions with human error into model training data for the sonar interpretation model in the model training and continuous optimization subsystem.

[0097] Specifically, sonar interpreters can register accounts on the data platform corresponding to the side device through the client, forming a personal learning profile (including basic level, historical assessment records and training preferences) and storing it in the personnel training and assessment subsystem. The personnel training and assessment subsystem will regularly provide sonar interpreters with modular courses such as theoretical explanation courses and simulation practical training through the client.

[0098] For example, theoretical courses can include sonar imaging principles, common interference types and parameter optimization, etc. Simulation and practical training can use simulators and historical real samples to support sonar interpreters to conduct task-based training (such as dragging parameters, target annotation, and noise scenario response) through intelligent training interactive all-in-one machines on client or side devices. The personnel training and assessment subsystem will record the response time and judgment results of each operation by the sonar interpreter, forming a unique learning progress indicator and personalized learning path for each sonar interpreter.

[0099] Specifically, regarding the aforementioned modular courses, after the course explanation is completed, the personnel training and assessment subsystem can also, on demand (i.e., based on the training and assessment needs of sonar interpreters), retrieve sonar images from the data warehouse that match the aforementioned training and assessment needs as training and assessment materials (equivalent to the scope of questions), thereby generating test questions for assessing the aforementioned sonar interpreters.

[0100] Step f2: In response to receiving the target assessment criteria, generate sonar interpretation assessment questions that match the target assessment criteria based on the training assessment materials, and distribute the sonar interpretation assessment questions to the client corresponding to each sonar interpreter.

[0101] Here, on one side of the side device, the staff responsible for assessing the sonar interpreters can input the assessment criteria (i.e., the target assessment criteria) into the personnel training and assessment subsystem. This prompts the subsystem to generate sonar interpretation assessment questions that match the target assessment criteria, within the currently locked question range (i.e., the training and assessment materials). The generated sonar interpretation assessment questions are then sent to the client corresponding to each sonar interpreter, automatically organizing all sonar interpreters to participate in the sonar interpretation assessment (equivalent to answering questions).

[0102] Step f3: Receive the response information from each client for the sonar interpretation test question, and select the incorrect response information from the received response information as the first misjudgment sample.

[0103] Here, at the end of the assessment period, the personnel training assessment subsystem can receive the answer information from each client regarding the sonar interpretation assessment questions (equivalent to automatically collecting the sonar interpretation assessment questions answered by each sonar interpreter at the end of the exam). From the received answer information, the system selects the answer information with incorrect answers (i.e. wrong questions) or with a high error rate (i.e., the proportion of incorrect answer information in all answer information is higher than the preset error rate) as the first misjudgment sample.

[0104] Step f4: Store the first misjudged sample as a candidate model training data for the sonar interpretation model in the data warehouse.

[0105] Here, the first misjudged sample is equivalent to the sample data corresponding to sonar images that sonar interpreters are prone to misinterpreting. Based on this, the personnel training and assessment subsystem can store the collected first misjudged sample as candidate model training data of the sonar interpretation model in the data warehouse, so that the model training and continuous optimization subsystem can subsequently obtain the above-mentioned candidate model training data as model training data or model optimization data (i.e., the aforementioned model fine-tuning training dataset) to train or optimize the sonar interpretation model.

[0106] Based on the sonar intelligent interpretation method provided in the embodiments of this application, this application integrates end-side equipment (deployed on a shipboard platform) and side-side equipment (deployed in a shore-based computer room) to form a sonar intelligent interpretation system. The sonar intelligent interpretation system realizes the fully automated processing of sonar interpretation from data acquisition to interpretation of sonar images, effectively improving the interpretation efficiency of sonar images. In addition, the sonar intelligent interpretation system can also simultaneously realize the model training of sonar interpretation models and the training and assessment of interpreters based on the massive sonar image data obtained from actual measurements (i.e., sonar images stored in the data warehouse), forming a sustainable closed-loop optimization system. This is conducive to solving many problems in traditional sonar interpretation, such as low interpretation accuracy, poor data reuse rate, and difficulty in personnel training and assessment.

[0107] Based on the same inventive concept, this application also provides a sonar intelligent interpretation system corresponding to the above-mentioned sonar intelligent interpretation method. Since the principle of solving the problem by the sonar intelligent interpretation system in the embodiments of this application is similar to that of the sonar intelligent interpretation method in the embodiments of this application, the implementation of the sonar intelligent interpretation system can refer to the implementation of the above-mentioned sonar intelligent interpretation method, and the repeated parts will not be described again.

[0108] Reference Figure 1 As shown, Figure 1 This illustration shows a structural diagram of a sonar intelligent interpretation system provided in an embodiment of this application. The sonar intelligent interpretation system includes: an end-side device and a side-side device, wherein the end-side device is deployed on a shipborne platform, and the side-side device is deployed in a shore-based engine room. A communication connection is established between the side-side device and the end-side device. The end-side device is used to classify and store the acquired raw sonar images into a data warehouse according to the data acquisition scenario corresponding to the sonar images; The side device is used to obtain sonar images matching the model training purpose from the data warehouse according to the model training purpose of the sonar interpretation model, as the model training dataset of the sonar interpretation model, and to train the sonar interpretation model using the model training dataset to obtain the trained target sonar interpretation model. The side-side device is used to evaluate and verify the target sonar interpretation model, and to send the target sonar interpretation model that has passed the evaluation and verification to the end-side device. The end-side device is used to receive and deploy the target sonar interpretation model, input the acquired original sonar image into the target sonar interpretation model, and obtain the first sonar interpretation result output by the target sonar interpretation model for the original sonar image.

[0109] In an optional implementation, the side device is further used for: Based on the training and assessment needs of sonar interpreters, sonar images matching the training and assessment needs are obtained from the data warehouse as training and assessment materials. In response to receiving the target assessment criteria, the system generates sonar interpretation assessment questions that match the target assessment criteria based on the training assessment materials, and distributes the sonar interpretation assessment questions to the client corresponding to each sonar interpreter. Receive the response information from each client in response to the sonar interpretation test questions, and filter out the incorrect response information from the received response information as the first misjudgment sample; The first misjudged sample is stored in the data warehouse as a candidate model training data for the sonar interpretation model.

[0110] In one optional implementation, when classifying and storing the acquired raw sonar images in a data warehouse according to the data acquisition scenario corresponding to the sonar images, the edge device is used to: In response to the triggering of preset conditions, the original sonar image is automatically acquired, and environmental parameters related to the original sonar image are recorded simultaneously; wherein, the environmental parameters include: timestamp parameters, platform attitude parameters of the shipborne platform, and underwater environmental parameters at the sonar image acquisition location; Based on the environmental parameters, a target data acquisition scene matching the environmental parameters is determined from multiple data acquisition scenes corresponding to the sonar image; The original sonar image is stored in the sonar image dataset corresponding to the target data acquisition scene in the data warehouse.

[0111] In one optional implementation, during the model evaluation and verification of the target sonar interpretation model, the side-side device is used to: From the data warehouse, obtain sonar images that match the model training objective as the model test dataset; The model test dataset is input into the target sonar interpretation model to obtain the second sonar interpretation result output by the target sonar interpretation model for the model test dataset; Based on the second sonar interpretation result and the model test data, calculate the index values ​​corresponding to multiple quantitative indicators used to evaluate the model performance of the target sonar interpretation model, and determine the model evaluation result of the target sonar interpretation model based on the index values ​​corresponding to the multiple quantitative indicators. Based on the simulation data and measured data corresponding to the sonar images in the preset data acquisition scenario, the target sonar interpretation model is verified in a scenario-based manner to obtain the model verification result of the target sonar interpretation model; wherein, the simulation data is a simulated sonar image generated for the preset data acquisition scenario, and the measured data is a sonar image obtained from the data warehouse that matches the preset data acquisition scenario.

[0112] In an optional implementation, when multiple end-side devices are deployed on the shipboard platform, when the target sonar interpretation model, which has been evaluated and verified by the model, is sent to the end-side devices: The side-side device is used to distribute the target sonar interpretation model to a preset number of end-side devices in response to the target sonar interpretation model passing the model evaluation and verification; wherein the preset number is less than the number of devices in the multiple end-side devices; The preset number of edge devices are used to, in response to receiving the target sonar interpretation model, replace the deployed old version sonar interpretation model with the target sonar interpretation model, monitor the online performance of the target sonar interpretation model in real time, and determine the online performance result of the target sonar interpretation model; wherein, the old version sonar interpretation model is the sonar interpretation model issued by the edge device at a historical time.

[0113] In one optional implementation, the preset number of end-side devices are further used for: In response to the online performance results not meeting the preset online performance standards of the model, the target sonar interpretation model is automatically switched to the old version of the sonar interpretation model; From the online performance data of the target sonar interpretation model, obtain the sonar image in which the sonar interpretation result of the target sonar interpretation model is incorrect as the second misjudgment sample, and send the second misjudgment sample to the side device; The side device is also used for: In response to receiving the second misjudged sample, the second misjudged sample is used as the model fine-tuning training dataset, and the target sonar interpretation model is fine-tuned and trained according to the model fine-tuning training dataset to obtain the fine-tuned target sonar interpretation model as the new version of the sonar interpretation model. The new version of the sonar interpretation model is distributed to the preset number of end-side devices.

[0114] In one optional implementation, when the acquired raw sonar image is input into the target sonar interpretation model to obtain a first sonar interpretation result output by the target sonar interpretation model for the raw sonar image, the end-side device is used to: The original sonar image is preprocessed to obtain a preprocessed original sonar image as the sonar image to be detected; The sonar image to be detected is input into the target sonar interpretation model. The target sonar interpretation model performs target detection and recognition on the sonar image to be detected, and outputs the target detection and recognition result as the first sonar interpretation result. The target detection and recognition result includes: the image position of the detected target in the sonar image to be detected, the target type to which the detected target belongs, and the confidence level of the detected target belonging to the target type.

[0115] In one optional implementation, the end-side device is further configured to: The BeiDou positioning information of the shipborne platform is obtained through the BeiDou module installed on the shipborne platform. Using an optical character recognition model, multiple key parameters are extracted from the sonar image to be detected; wherein, the multiple key auxiliary parameters are parameters relating to the relative position between the target and the shipborne platform. From the target detection and recognition results, the image position of the detected target is obtained, and the image position is fused with the Beidou positioning information and the multiple key parameters to obtain the latitude and longitude position and relative orientation information of the detected target as the first sonar interpretation result; wherein, the relative orientation information is the relative orientation information between the detected target and the shipborne platform.

[0116] In one optional implementation, the end-side device is further configured to: Multiple frames of historical sonar images continuously acquired within the previous time period are input into the target sonar interpretation model to obtain the historical sonar interpretation results output by the target sonar interpretation model for the multiple frames of historical sonar images; wherein, the historical sonar interpretation results include: multiple historical detection targets identified from the multiple frames of historical sonar images; The latest sonar image acquired within the current time period is input into the target sonar interpretation model to obtain the latest sonar interpretation result output by the target sonar interpretation model for the latest sonar image; wherein, the latest sonar interpretation result includes: the latest detection target identified from the latest sonar image; Based on the latest sonar interpretation results and the historical sonar interpretation results, determine whether the number of times the latest detected target appears in the multi-frame historical sonar images is greater than or equal to a preset number threshold. If it is determined that the number of occurrences is greater than or equal to the preset number threshold, then in the latest sonar interpretation result, the latest detected target is marked as the real detected target; If it is determined that the number of occurrences is less than the preset number threshold, then in the latest sonar interpretation results, the latest detected target is marked as a suspected detected target.

[0117] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0118] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0119] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A sonar intelligent interpretation method, characterized in that, An application is made in a sonar intelligent interpretation system, the sonar intelligent interpretation system comprising: an end-side device and a side-side device, wherein the end-side device is deployed on a shipborne platform, and the side-side device is deployed in a shore-based engine room, a communication connection is established between the side-side device and the end-side device, and the sonar intelligent interpretation method comprises: The end-side device classifies and stores the acquired raw sonar images into a data warehouse according to the data acquisition scenario corresponding to the sonar images; According to the model training objective of the sonar interpretation model, the side device obtains sonar images that match the model training objective from the data warehouse as the model training dataset of the sonar interpretation model, and trains the sonar interpretation model using the model training dataset to obtain the trained target sonar interpretation model. The side-side device performs model evaluation and verification on the target sonar interpretation model, and sends the target sonar interpretation model that has passed the model evaluation and verification to the end-side device; The end-side device receives and deploys the target sonar interpretation model, inputs the acquired original sonar image into the target sonar interpretation model, and obtains the first sonar interpretation result output by the target sonar interpretation model for the original sonar image.

2. The sonar intelligent interpretation method according to claim 1, characterized in that, The sonar intelligent interpretation method also includes: The side device retrieves sonar images matching the training and assessment requirements of sonar interpreters from the data warehouse as training and assessment materials. In response to receiving the target assessment standard, the side device generates sonar interpretation assessment questions that match the target assessment standard based on the training assessment materials, and sends the sonar interpretation assessment questions to the client corresponding to each sonar interpreter. The side device receives the answer information from each client in response to the sonar interpretation test questions, and filters out the answer information with incorrect answers from the received answer information as the first misjudgment sample; The side device stores the first misjudged sample as a candidate model training data for the sonar interpretation model in the data warehouse.

3. The sonar intelligent interpretation method according to claim 1, characterized in that, The step of classifying and storing the acquired raw sonar images in a data warehouse according to the data acquisition scenario corresponding to the sonar images includes: The end-side device, in response to the triggering of preset conditions, automatically acquires the original sonar image and simultaneously records the environmental parameters related to the original sonar image; wherein, the environmental parameters include: timestamp parameters, platform attitude parameters of the shipborne platform, and underwater environmental parameters at the sonar image acquisition location; The end-side device determines a target data acquisition scene that matches the environmental parameters from multiple data acquisition scenes corresponding to the sonar image; The end-side device stores the original sonar image into the sonar image dataset corresponding to the target data acquisition scene in the data warehouse.

4. The sonar intelligent interpretation method according to claim 1, characterized in that, The model evaluation and validation of the target sonar interpretation model includes: The side device retrieves sonar images that match the model training objective from the data warehouse as the model test dataset; The side device inputs the model test dataset into the target sonar interpretation model to obtain the second sonar interpretation result output by the target sonar interpretation model for the model test dataset; The side device calculates the index values ​​corresponding to multiple quantitative indicators for evaluating the model performance of the target sonar interpretation model based on the second sonar interpretation result and the model test data, and determines the model evaluation result of the target sonar interpretation model based on the index values ​​corresponding to the multiple quantitative indicators. The side-side device performs scenario-based verification of the target sonar interpretation model based on the simulation data and measured data corresponding to the sonar image in the preset data acquisition scenario, and obtains the model verification result of the target sonar interpretation model; wherein, the simulation data is a simulated sonar image generated for the preset data acquisition scenario, and the measured data is a sonar image obtained from the data warehouse that matches the preset data acquisition scenario.

5. The sonar intelligent interpretation method according to claim 1, characterized in that, When multiple end-side devices are deployed on the shipborne platform, the step of sending the target sonar interpretation model, which has been evaluated and verified by the model, to the end-side devices includes: In response to the target sonar interpretation model being evaluated and verified by the model, the side-side device distributes the target sonar interpretation model to a preset number of end-side devices; wherein the preset number is less than the number of devices in the multiple end-side devices. The preset number of edge devices, in response to receiving the target sonar interpretation model, replace the deployed old version sonar interpretation model with the target sonar interpretation model, and monitor the online performance of the target sonar interpretation model in real time to determine the online performance result of the target sonar interpretation model; wherein, the old version sonar interpretation model is the sonar interpretation model issued by the edge device at a historical time.

6. The sonar intelligent interpretation method according to claim 5, characterized in that, The sonar intelligent interpretation method also includes: When the online performance results do not meet the preset model online performance standards, the preset number of end-side devices automatically switch the target sonar interpretation model to the old version of the sonar interpretation model. The preset number of edge devices obtain sonar images from the online performance data of the target sonar interpretation model where the sonar interpretation result of the target sonar interpretation model is incorrect as second misjudgment samples, and send the second misjudgment samples to the edge devices; In response to receiving the second misjudged sample, the side device uses the second misjudged sample as the model fine-tuning training dataset, and fine-tunes the target sonar interpretation model according to the model fine-tuning training dataset to obtain the fine-tuned target sonar interpretation model as the new version of the sonar interpretation model. The side-side device distributes the new sonar interpretation model to the preset number of end-side devices.

7. The sonar intelligent interpretation method according to claim 1, characterized in that, The step of inputting the acquired raw sonar image into the target sonar interpretation model to obtain the first sonar interpretation result output by the target sonar interpretation model for the raw sonar image includes: The end-side device preprocesses the original sonar image to obtain a preprocessed original sonar image as the sonar image to be detected. The end-side device inputs the sonar image to be detected into the target sonar interpretation model, performs target detection and recognition on the sonar image to be detected through the target sonar interpretation model, and outputs the target detection and recognition result as the first sonar interpretation result; wherein, the target detection and recognition result includes: the image position of the detected target in the sonar image to be detected, the target type to which the detected target belongs, and the confidence level of the detected target belonging to the target type.

8. The sonar intelligent interpretation method according to claim 1, characterized in that, The sonar intelligent interpretation method also includes: The end-side device obtains the BeiDou positioning information of the shipborne platform through the BeiDou module installed on the shipborne platform; The end-side device extracts multiple key parameters from the sonar image to be detected using an optical character recognition model; wherein, the multiple key auxiliary parameters are parameters relating to the relative position between the detection target and the shipborne platform; The end-side device obtains the image position of the detected target from the target detection and recognition results, and fuses the image position with the Beidou positioning information and the multiple key parameters to obtain the latitude and longitude position and relative orientation information of the detected target as the first sonar interpretation result; wherein, the relative orientation information is the relative orientation information between the detected target and the shipborne platform.

9. The sonar intelligent interpretation method according to claim 1, characterized in that, The sonar intelligent interpretation method also includes: The end-side device inputs multiple frames of historical sonar images continuously acquired within the previous time period into the target sonar interpretation model to obtain the historical sonar interpretation results output by the target sonar interpretation model for the multiple frames of historical sonar images; wherein, the historical sonar interpretation results include: multiple historical detection targets identified from the multiple frames of historical sonar images; The end-side device inputs the latest sonar image acquired within the current time period into the target sonar interpretation model to obtain the latest sonar interpretation result output by the target sonar interpretation model for the latest sonar image; wherein, the latest sonar interpretation result includes: the latest detection target identified from the latest sonar image; The end-side device determines, based on the latest sonar interpretation results and the historical sonar interpretation results, whether the number of times the latest detected target appears in the multi-frame historical sonar images is greater than or equal to a preset number threshold. If the end-side device determines that the number of occurrences is greater than or equal to the preset number threshold, then in the latest sonar interpretation result, the latest detected target is marked as a real detected target; If the end-side device determines that the number of occurrences is less than the preset number threshold, then in the latest sonar interpretation result, the latest detected target is marked as a suspected detected target.

10. A sonar intelligent interpretation system, characterized in that, The sonar intelligent interpretation system includes: an end-side device and a side-side device, wherein the end-side device is deployed on a shipboard platform, and the side-side device is deployed in a shore-based engine room, and a communication connection is established between the side-side device and the end-side device. The end-side device is used to classify and store the acquired raw sonar images into a data warehouse according to the data acquisition scenario corresponding to the sonar images; The side device is used to obtain sonar images matching the model training purpose from the data warehouse according to the model training purpose of the sonar interpretation model, as the model training dataset of the sonar interpretation model, and to train the sonar interpretation model using the model training dataset to obtain the trained target sonar interpretation model. The side-side device is used to evaluate and verify the target sonar interpretation model, and to send the target sonar interpretation model that has passed the evaluation and verification to the end-side device. The end-side device is used to receive and deploy the target sonar interpretation model, input the acquired original sonar image into the target sonar interpretation model, and obtain the first sonar interpretation result output by the target sonar interpretation model for the original sonar image.

Images