A dynamic quantitative completeness test method and device for a ship intelligent collision avoidance algorithm

Abstract

This invention provides a method and apparatus for dynamically quantifying the completeness of intelligent collision avoidance algorithms for ships, relating to the field of ship navigation safety technology. This method addresses the full-process verification of intelligent collision avoidance algorithms, based on a fundamental model and algorithm for ship-anthropomorphic intelligent collision avoidance decision-making. Through a full-link capability encompassing "constructing a complete scenario – algorithm closed-loop testing – quantitative evaluation of indicators – intelligent report generation," it achieves objective and efficient verification of the completeness of collision avoidance algorithms, providing core support for algorithm development iteration, compliance acceptance, and real-ship application. Its functions focus on three core objectives: "systematization and dynamization of evaluation standards, completeness of scenario coverage, and automation of the testing and evaluation process."

Description

Technical Field

[0001] This invention relates to the field of ship navigation safety technology, specifically to a method and apparatus for dynamic quantitative completeness testing of ship intelligent collision avoidance algorithms. Background Technology

[0002] Intelligent collision avoidance algorithms for ships are core technologies for realizing intelligent navigation and autonomous decision-making. The completeness of these algorithms directly affects navigation safety and shipping efficiency. In recent years, with the rapid development of intelligent ship technology, research on the testing and evaluation system of intelligent collision avoidance algorithms has become a hot topic in this field. Currently, some progress has been made in algorithm evaluation methods. For example, Sun Feng et al. proposed a testing and evaluation method for intelligent collision avoidance strategies based on the International Regulations for Preventing Collisions at Sea (COLREGs), which comprehensively considered the evaluation indicators from a qualitative perspective. Meanwhile, the national standard "Technical Requirements and Test Methods for Intelligent Ship Collision Avoidance Systems" compiled by Cai Yanxian et al. also provides a basic framework for hazard assessment threshold references, encounter situation classification methods, and scenario design. Building on this, Luo Yanwen et al. further conducted research on the evaluation method of intelligent collision avoidance algorithms for two ships in open waters, constructed a model including an evaluation indicator system, and initially developed an automatic scoring system. The specific implementation process of the automatic scoring system is as follows: First, the data required for calculating the evaluation functions of each indicator after the scenario is run is processed by the data processing system and then input into the scoring system, which is equipped with an evaluation model and a reference answer library. The reference answer library is established by analyzing the "Rules," seafarers' common practices, and existing dynamic assessment and collision avoidance decision-making models for ship collision risks. After scoring by the scoring system, the data is input into the results output interface, and finally, the scores and results for each indicator are output. The modules of the automatic evaluation system include... Figure 1 As shown.

[0003] However, the aforementioned existing technical solutions still have significant limitations and cannot meet the actual needs of testing the completeness of intelligent collision avoidance algorithms for ships. First, existing research mainly focuses on yielding vessels in typical single-target vessel encounter situations. Its evaluation index functions and standards do not cover scenarios involving straight-going vessels, multi-target vessel encounters, or extreme conditions such as uncoordinated, out-of-control, or restricted target vessel behavior. This results in a narrow scope of evaluation, failing to comprehensively reflect the algorithm's true performance in complex encounter situations. Second, regarding test scenario construction, existing solutions only consider a few typical scenarios. There is a lack of a standardized scenario library and its corresponding quantitative reference answer library that systematically covers single-target, multi-target, and various special cases to support algorithm completeness testing, resulting in insufficient comprehensiveness and repeatability of the tests.

[0004] More critically, existing automated evaluation systems essentially rely on static comparisons using a pre-set reference answer library, lacking real-time dynamic quantitative evaluation capabilities based on ship collision avoidance geometry models. Furthermore, their evaluation criteria use fixed thresholds, failing to dynamically adapt to different ship sizes, encounter situations, avoidance responsibilities, and hazard levels. These technical deficiencies collectively make it difficult for existing testing systems to adapt to the complex and ever-changing temporal evolution of maritime encounter scenarios. They are unable to conduct a systematic, objective, and quantitative comprehensive assessment of the compliance, safety, and economics of collision avoidance algorithms, thus becoming a core technical bottleneck restricting the research, verification, and industrial application of intelligent ship collision avoidance decision-making algorithms.

[0005] In view of the above, this application is hereby submitted. Summary of the Invention

[0006] This invention provides a method and apparatus for dynamic quantitative completeness testing of ship intelligent collision avoidance algorithms, which can at least partially improve the above-mentioned problems.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A method for dynamic quantitative completeness testing of intelligent collision avoidance algorithms for ships, comprising: A dynamic and quantitative scenario-based scoring rule is constructed, which includes multi-level indicators in three dimensions: safety, compliance, and economy. The evaluation thresholds for each indicator can be dynamically adjusted according to the ship's size, encounter situation, and hazard level. A comprehensive test scenario library is constructed, which includes a single-target ship scenario library, a two-target ship scenario library, and a special scenario library, and quantitative reference answers for each secondary indicator are preset for each scenario library; A ship automatic collision avoidance simulation test platform was established based on scenario-based scoring rules and a complete test scenario library. The ship automatic collision avoidance simulation test platform was used to conduct simulation tests on each scenario in the complete test scenario library to obtain the first test report. Obtain historical AIS data of actual ships in the selected waters, and use the ship automatic collision avoidance simulation test platform to conduct simulation tests on the historical AIS data of actual ships to obtain a second test report; Based on the first and second test reports, attribution analysis and clustering are performed on the failure scenarios that occurred during the test, and targeted enhancement test scenarios are automatically generated and added to the completeness test scenario library. At the same time, a multi-dimensional quantitative completeness assessment report is generated. For the supplemented and complete test scenario library, continue to use the ship automatic collision avoidance simulation test platform for simulation testing, and repeat the evaluation until all scenarios in the library pass the test.

[0009] This invention also provides a device for testing the dynamic quantification completeness of ship intelligent collision avoidance algorithms, comprising: The rule building unit is used to build dynamic and quantitative scenario-based scoring rules. The scenario-based scoring rules include multi-level indicators in three dimensions: safety, compliance and economy. The evaluation threshold of each indicator can be dynamically adjusted according to the ship size, encounter situation and hazard level. The scenario library construction unit is used to construct a completeness test scenario library, which includes a single-target ship scenario library, a two-target ship scenario library, and a special scenario library, and presets quantitative reference answers for each secondary indicator for each scenario library; The first test unit is used to establish a ship automatic collision avoidance simulation test platform based on scenario-based scoring rules and a complete test scenario library. The ship automatic collision avoidance simulation test platform is used to conduct simulation tests on each scenario in the complete test scenario library to obtain the first test report. The second test unit is used to acquire historical AIS data of actual ships in selected waters, and to conduct simulation tests on the historical AIS data of actual ships using a ship automatic collision avoidance simulation test platform to obtain a second test report. The optimization unit is used to perform attribution analysis and clustering of failure scenarios that occur during the test based on the first test report and the second test report, automatically generate targeted enhancement test scenarios and supplement them to the completeness test scenario library, and generate a multi-dimensional quantitative completeness evaluation report. The iterative unit is used to continue to conduct simulation tests using the ship automatic collision avoidance simulation test platform for the supplemented completeness test scenario library, and to repeatedly evaluate it until all scenario libraries have passed the test.

[0010] In summary, this method, focusing on the full-process verification of intelligent collision avoidance algorithms for ships, is based on a fundamental model and algorithm for ship-anthropomorphic intelligent collision avoidance decision-making. Through a complete chain capability encompassing "constructing a comprehensive scenario – algorithm closed-loop testing – quantitative evaluation of indicators – intelligent report generation," it achieves objective and efficient verification of the completeness of collision avoidance algorithms, providing core support for algorithm R&D iteration, compliance acceptance, and real-ship application. Addressing the completeness verification needs of intelligent collision avoidance algorithms for ships, this method constructs a multi-dimensional algorithm completeness evaluation, achieving standardized and precise evaluation of algorithm testing.

[0011] Compared with existing technologies, this method has the following beneficial effects: 1. Technical level: Improved evaluation accuracy. This invention constructs a multi-dimensional evaluation system, including compliance indicators (whether it conforms to the priority of COLREGs rules), safety indicators, and economic indicators, to achieve multi-dimensional completeness evaluation of the algorithm and improve evaluation accuracy. Considering multiple dimensions such as coverage of the International Maritime Collision Prevention Regulations (IMDG), geometric characteristics of target encounters, common single-target encounters, possible different encounter modes between two target vessels, possible target vessel incoordination, loss of control, and emergency situations, an algorithm completeness scenario library is constructed, covering the full capability boundary of the algorithm, enabling algorithm completeness testing and improving algorithm reliability. 2. Application level: Adapted to actual ship collision avoidance needs, ensuring navigation safety. This invention not only meets the testing requirements of a targeted completeness scenario library but also fits real-world maritime scenarios, improving the practicality of the algorithm. A standardized evaluation process is proposed to promote the standardization of algorithm testing. A standard process for collision avoidance algorithm completeness testing is established, clarifying test data formats, scenario classification rules, and evaluation indicator thresholds, solving the problem of inconsistent testing methods and incomparable results among different R&D entities. The generated test reports can be directly used for algorithm achievement acceptance, patent application, or technical appraisal, providing authoritative evaluation basis for the industrial application of intelligent ship collision avoidance systems. 3. Industry Level: Promoting the large-scale implementation of intelligent ship collision avoidance technology. Firstly, it lowers the barrier to algorithm development: providing an out-of-the-box automated testing toolchain covering the entire process from scenario library, data (hazard assessment threshold) driven, indicator evaluation, to report output, eliminating the need for R&D teams to develop additional testing systems and reducing the technology R&D costs for SMEs. Secondly, it supports intelligent ship compliance certification: meeting the requirements of maritime regulatory authorities for the safety and compliance certification of intelligent collision avoidance algorithms. Through the standardized test data and evaluation reports generated by the system, algorithm qualification review can be completed quickly, accelerating the transformation of technological achievements. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the modules of an automated evaluation system provided by existing technology.

[0013] Figure 2 This is a flowchart illustrating a method for dynamic quantification of the completeness of a ship intelligent collision avoidance algorithm, as provided in the first embodiment of the present invention.

[0014] Figure 3 This is a schematic diagram of the process framework of a dynamic quantitative completeness test method for ship intelligent collision avoidance algorithm provided in the first embodiment of the present invention.

[0015] Figure 4 This is a schematic diagram of the encounter situation, encounter attributes, and division of avoidance responsibilities based on the relative bearing of the approaching vessel, provided by an embodiment of the present invention.

[0016] Figure 5This is a schematic diagram of the geometric model of the ship collision hazard and hazard level assessment threshold index provided in the embodiments of the present invention.

[0017] Figure 6 This is a schematic diagram of the VICAD algorithm evaluation index system provided in the embodiments of the present invention.

[0018] Figure 7 This is a schematic diagram of the quantitative geometric model of the reversal collision avoidance scheme provided in the embodiment of the present invention.

[0019] Figure 8 This is a schematic diagram of the evaluation standard system and dynamic adjustment rules for the two-level indicators of safety, compliance and economy provided in the embodiments of the present invention.

[0020] Figure 9 This is a schematic diagram of the automatic evaluation system for the completeness test of intelligent collision avoidance algorithms for ships provided in an embodiment of the present invention.

[0021] Figure 10 This is a simulation test and automatic evaluation result diagram of three-ship encounter multi-intelligent-ship PIDVCA provided in an embodiment of the present invention, wherein the encounter situation is two intelligent ships.

[0022] Figure 11 This is a simulation test and automatic evaluation result diagram of three ships encountering multiple intelligent ships using PIDVCA provided in an embodiment of the present invention, wherein all three ships are intelligent ships.

[0023] Figure 12 This is a schematic diagram of a module of a ship intelligent collision avoidance algorithm dynamic quantification completeness testing device provided in the second embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0025] refer to Figure 2 , Figure 3 As shown, the first embodiment of the present invention discloses a method for testing the dynamic quantification completeness of a ship intelligent collision avoidance algorithm, which can be executed by a ship intelligent collision avoidance algorithm dynamic quantification completeness testing device (hereinafter referred to as the testing device), specifically, by one or more processors within the testing device, to implement the following method: S1. Construct dynamic and quantitative scenario-based scoring rules. The scenario-based scoring rules include multi-level indicators in three dimensions: safety, compliance, and economy. The evaluation thresholds of each indicator can be dynamically adjusted according to the ship size, encounter situation, and hazard level. Specifically, step S1 further includes: constructing a ship encounter situation and attribute identification model based on preset encounter information, including the encounter situation of the relative bearing of the approaching ship, encounter attributes, and division of avoidance responsibilities; The hazard assessment is based on the ideal safe encounter distance, the critical safe encounter distance, and the critical collision distance. Based on the geometric analysis of the relative motion of the ships, a quantitative model of the threshold for each hazard level is established. The critical safe encounter distance and the critical collision distance are dynamically calculated based on the lengths of the ship and the target ship, the angle between the headings of the two ships, and the turning performance parameters of the ship. Based on the collision avoidance decision parameters, a collision avoidance parameter quantification model is constructed. The collision avoidance decision parameters include the optimal timing for taking collision avoidance action, the magnitude of the action, and the quantification method for predicting the rerouting point timing. The optimal timing and the magnitude of the action are dynamically calculated based on the current relative distance, relative speed, the turning performance of the ship itself, and the motion parameters of the target ship. Based on the ship encounter situation and attribute identification model, the threshold quantification model for each hazard level, and the collision avoidance parameter quantification model, the analytic hierarchy process is used to generate scenario-based scoring rules. The threshold values ​​of each indicator used in the scenario-based scoring rules are dynamically adjusted according to the ship size and hazard level. The scenario-based scoring rules include three primary indicators and eight secondary indicators. The primary indicators are the criteria, and the secondary indicators are the indicators. The three primary indicators are safety, compliance, and economy. The eight secondary indicators are: two secondary indicators under safety, namely collision hazard assessment threshold and hazard level classification; five secondary indicators under compliance, namely encounter situation and liability judgment, avoidance action method and direction, avoidance action timing, avoidance action range, and avoidance action frequency; and one secondary indicator under economy, namely flight deviation.

[0026] In this embodiment, a VICAD evaluation index system is constructed based on the International Regulations for Preventing Collisions at Sea (ICCMS), a questionnaire of maritime experts' experience, and a collision avoidance geometry model. Figure 6 This involves establishing a multi-level, quantifiable, and dynamic indicator threshold evaluation standard across the three dimensions of security, compliance, and economy (as shown). The core of this step is to establish a multi-level, quantifiable, comprehensive evaluation indicator system to provide a precise "benchmark" for subsequent automated testing. This system not only includes primary and secondary indicators, but more importantly, it sets clear quantitative models and calculation criteria for each indicator, and can dynamically adjust indicator thresholds to adapt to evaluation standards for different levels of testing scenarios.

[0027] Specifically, based on Figure 4 The ship encounter situation and attribute recognition model constructed with Table 1 can identify the encounter situation.

[0028] Table 1: Identification Model for Encounter Situations, Encounter Attributes, and the Vessel's Liability to Avoid Collisions (See each other)

[0029] Immediately afterwards, based on the associated avoidance behavior, a geometric model of the thresholds of ship collision risk and risk level assessment indicators is established (as shown in Figure 5 ); the quantification of the risk assessment thresholds (as shown in Table 2) and risk level thresholds (as shown in Table 3) in the secondary indicators, as well as the evaluation criteria for the optimal avoidance timing, is achieved. Among them, the optimal avoidance behavior (as shown in Table 4) and the risk assessment thresholds are from the processing results of questionnaires of 400 experienced drivers on the front line; Table 5 shows the processing results of questionnaires for bulk carriers with a ship length of 100 meters (80 < Lo < 150).

[0030] Table 2: Names and Meanings of Ship Collision Risk Assessment Thresholds

[0031] Table 3: Names and Meanings of Ship Collision Risk Degree Assessment Thresholds

[0032] Table 4: Questionnaire Results of Common Course-changing Angles for Typical Encounter Situations in Different Waters

[0033] Table 5: SDA of 100m Bulk Carrier c , pref , min , col , c , pref , min , col , c , col , c , col , ac , pref Table (unit: nmile)

[0034] Specifically, as can be seen from Figure 6 , the risk assessment consists of three thresholds: the ideal safe encounter distance SDA pref , the critical safe encounter distance SDA c , and the critical collision distance DA col . DA col depends on the ship size, the course intersection angle of the encounter between two ships, and the geometric relationship formed by just not colliding from the bow or stern of the other ship. The error of the radar position is also considered in the relationship. Among them, the ideal safe encounter distance SDA pref , the critical safe encounter distance SDA c are quantitatively calculated by the formulas SDA c = DA col + M min and SDA pref = SDA c + M ac ; M min is the minimum safety margin for the two ships to pass safely while maintaining their courses and speeds without changing course. The value is obtained from the original collision critical distance questionnaire data of 3 types of own ship models, 4 types of target ship models, and 22 typical situations and the corresponding DA colExtract the pilot experience data for each typical situation; the ship maneuvering margin Mac is quantitatively calculated by a geometric model built from the angle between the two ships' headings, the ship's turning and turning distance, the maneuvering delay t, and the target ship's speed.

[0035] based on Figure 5 Based on the geometrical analysis of the relative motion of ships (taking a right-hand crossover as an example), a quantitative model for the threshold of each danger level in Table 3 is established. The imminent situation distance DCQS (AC) is used as the benchmark. m SDA c Taking (e.g., RML and NRML2) as an example, let's list the equations of the two lines respectively. Their formulas are: , where C r With C rn These represent the headings of the target vessel relative to the ship before and after the course change. sgn(SDA) is the sign function of SDA. The value of sgn(SDA) mainly depends on the target intersection characteristic (TEC) and the magnitude and direction of the course change. When the ship yields to AC on either starboard or port, the course change is determined based on the changing pattern of the target vessel's relative motion line and its C. rn The possible range of values ​​determines the value of sgn(SDA). In this example, sgn(SDA) = -1.

[0036] To calculate the relative heading C of the target ship after the ship changes course rn The relative velocity V must be calculated first. rn Therefore, AC is defined as the change-of-course magnitude. The ship's new relative velocity V is determined by the change-of-course avoidance maneuver. rn and heading C rn The calculation formula is as follows: , .

[0037] AC m Substitute AC into the formula above, and calculate C. rn Substituting into the equation of the two lines, we obtain the intersection point N(x) of the two lines. b y b ),get .

[0038] In open water, the relative displacement SS can be calculated by turning 90° to starboard at full speed and with full rudder. Its components along the x and y axes are then obtained as x. s With y s ,get , , among which, T n D is the turning time for the ship to turn 90° at full speed and with full rudder. c For the initial diameter of the cycle, F d V is the gyration advance distance; t and C t These represent the target ship's true speed and true course, respectively; Kt These are the proportional coefficients for ship maneuvering parameters at different course change angles, with values ​​less than 1.

[0039] Therefore, take a point L on the RML such that the length of LN is equal to SS, and use L as the estimated avoidance execution point L(x). a y a The value of ) is given by the formula Calculation. Therefore, the urgent situation is far from DCQS (AC). m SDA c (DCQS) is Therefore, it can be deduced that reaching L(x) a y a The best time to avoid (T) ln , V r Let T be the relative velocity of the target ship. Obviously, from the formula... ln The calculated value is T. ln (AC m SDA c When V in the above formula rn (AC) and C rn In (AC), AC is represented by AC respectively. i and AC m And SDA pref Use DA col Substitute the values ​​and apply them to the component formulas of AC. t For different values ​​of , the optimal distance to the avoidance action execution point (DCR) can be obtained from the above formula. c (AC i SDA pref ) and DID c (AC m DA col ) and the T of the target ship reaching the corresponding position point isr (AC i SDA pref ), T isr (AC m SDA pref ) and T ln (AC m DA col ). T isr (AC i SDA pref ) and T ln (AC m Data such as SDA (Survey Data Allocation) can objectively reflect the urgency of the collision risk to the target vessel, and also include the ease or difficulty of the vessel itself to avoid the dangerous vessel. They can objectively reflect the magnitude of the potential collision risk between the dangerous target vessel and the vessel in time and space.

[0040] Therefore, the criteria for classifying critical collision hazard, critical imminent situation, imminent situation, and critical imminent hazard are as follows: when T ln ∈{ T isr (AC i SDA pref )<0,T ln (AC m SDA pref When T ≥ 0, the two ships are determined to be in a critical collision situation; when T ln ∈{ T ln (AC m SDA pref )<0,T ln (AC m SDA c When T ≥ 0, the two ships are determined to be in a critical and urgent situation; when T ln ∈{T ln (AC m SDA c )<0,T ln (AC m DA col When T ≥ 0, it is determined that the two ships are in a tense situation; when T ln (AC m DA col If the value is less than 0, then both ships are considered to be in critical danger.

[0041] Please see Figure 7 The parameters for collision avoidance decision-making include the optimal timing for taking collision avoidance action, the magnitude of the action, and quantitative methods for predicting the re-entry point. Figure 7 The left half is the optimal course change for this vessel (AC). i (Table 4 shows that the angle in this example is 30°) Avoidance reference target ship's avoidance rudder point P a Turning point P b and predicted re-entry point P r A schematic diagram, Figure 7 The right half of the map represents the point where the ship missed the optimal steering point P. a Immediately change direction at current position C, then turn at point AC. p and predicted re-entry point P r A schematic diagram.

[0042] When this ship misses AC i In order to calculate the turning angle AC of the ship to avoid the target ship, based on Figure 7 Turning point AC p (x) p y p ), Predicted relative motion heading C for avoiding key vessels rn The calculation model is R p sgn(SDA) represents the relative distance between the vessel and the reference target vessel. sgn(SDA) is the SDA sign function, which depends on the target rendezvous characteristic (TEC) and the vessel's change direction. In this example, when the vessel changes direction to starboard, sgn(SDA) = 1. Predicted turning point. The calculation model is , T(AC) is the maneuvering delay for the ship's course change AC, based on T ln Determine the size and estimate the delay.

[0043] According to C rn and Figure 7 Given the velocity vector ΔABC, the mathematical model for solving AC can be derived using the sine theorem. Using the analytic hierarchy process (AHP), based on the importance of each indicator to navigation safety and efficiency, its weight in the comprehensive evaluation is scientifically determined, forming a set of scenario-based scoring rules that can be automatically executed and output quantitative scores, such as... Figure 8 As shown.

[0044] The constructed dynamic quantitative scenario-based scoring rules are shown in Table 6.

[0045] Table 6:

[0046] S2, construct a complete test scenario library, which includes a single-target ship scenario library, a two-target ship scenario library, and a special scenario library, and pre-set quantitative reference answers for each secondary indicator for each scenario library; Specifically, step S2 further includes: constructing a single-target ship scenario library, which involves: fixing the ship type, course and speed of the ship, setting the target ship as an intelligent ship that complies with the International Maritime Collision Avoidance Rules, and determining 11 typical basic ship encounter situations according to the International Maritime Collision Avoidance Rules; For each basic ship encounter situation, based on 2 hazard levels, 3 test scenarios are designed. The 11 typical basic ship encounter situations are multiplied by the 3 test scenarios respectively to obtain 33 single-objective test scenarios. These are combined to generate a single-objective test scenario library, and quantitative reference answers for each secondary indicator are preset for the single-objective test scenario library. The three test scenarios are: Level 1 danger (test scenario where the best opportunity has not been missed), Level 1 danger (test scenario where the best opportunity has been missed), and Level 2 danger (test scenario).

[0047] A two-target ship scenario library and a special scenario library were constructed. Specifically, 11 single-target ship scenarios with level one danger and without missing the best opportunity were used as basic seed scenarios. A third target ship was introduced using conflict-type intervention mode and collaborative danger-type intervention mode, respectively, to generate 210 two-target ship test scenarios and a two-target test scenario library. Quantitative reference answers for each secondary indicator were preset for the two-target test scenario library. There are 100 scenarios for conflict-type intervention mode and 110 scenarios for collaborative danger-type intervention mode. Among them, the conflict-type intervention mode means that a third vessel appears on the avoidance path planned by this vessel to avoid the first target vessel, forming a new conflict. The collaborative danger-type intervention mode means that the third vessel and the first target vessel together pose a collision hazard to this vessel, forming a multiple threat. In 11 single-target vessel scenarios where the first level of danger has not been missed and the best opportunity has not been missed, the target vessel is set to take actions that do not comply with the rules, resulting in 11 corresponding special target vessel scenarios. Actions that do not comply with the rules include giving way vessels failing to fulfill their avoidance obligations and maintaining their course and speed or failing to exercise the right of straight-going vessels and taking uncoordinated actions. In the scenario of a left-hand cross encounter, the target vessel is set as a trawler, and in the scenario of being overtaken, the target vessel is set as an out-of-control vessel, thus creating two special target vessel scenarios. Under 11 typical ship encounter situations, two scenarios were set up: the ship passing by the bow of the target ship and the ship passing by the stern of the target ship. A total of 22 corresponding special target ship scenarios were generated to test the emergency avoidance capability of the algorithm. All the above-mentioned special target ship scenarios are summarized to generate a special scenario library, and quantitative reference answers for each secondary indicator are preset for the special scenario library.

[0048] In this embodiment, the construction of the completeness test scenario library of the present invention fully considers different hazard levels, the geometric laws of relative motion collision avoidance of ships, dangerous target ships that may appear in open waters, and the scenario requirements covered by the provisions of the Regulations. The collision avoidance algorithm test scenarios are divided into single-target scenario libraries, two-target scenario libraries, and special scenario libraries. This step determines the method for constructing the algorithm test completeness scenario library to achieve complete scenario coverage. Please refer to Table 7.

[0049] Table 7:

[0050] Specifically, the construction of the single-target test scenario library involves: first, determining the basic parameters, fixing the ship type, heading (0°), and speed (15 knots, which can be set based on the ship's normal speed), and setting the target ship as an intelligent ship that complies with the Regulations; based on the International Collision Avoidance Regulations, determining 11 typical basic ship encounter situations (as shown in Table 5); for each basic situation, considering 2 hazard levels, designing 3 test scenarios. Among them, "Level 1 Hazard" is further subdivided into two avoidance timing situations: Situation A (not missing the best opportunity), the avoidance action range is determined based on the results of a questionnaire survey of maritime experts; Situation B (missing the best opportunity), the avoidance action range is calculated by the collision avoidance geometry model, which usually requires a larger emergency avoidance. By combining the 11 basic situations × 3 situations (Level 1 Hazard A, Level 1 Hazard B, Level 2 Hazard), a total of 33 single-target test scenarios are generated, as shown in Table 8.

[0051] Table 8: Typical Test Scenarios for Single-Target Ships

[0052] S1–S8 represent the quantitative evaluation criteria for eight secondary indicators: the situation encountered, the timing of action, the method of action, the scope of action, the number of actions, the safe distance, the hazard level, and the track deviation. The target elements are shown in Table 8.1.

[0053] Table 8.1: Target Elements

[0054] For the construction of a two-target test scenario library, the aforementioned 11 single-target scenarios of "Level 1 Danger (Opportunity Not Missed)" are used as basic seed scenarios. Two typical intervention modes for the third ship are designed to increase decision-making complexity: Mode A: Conflict-type intervention; Mode B: Cooperative danger-type intervention. For Mode A (Conflict-type intervention): Based on the law of relative motion change, the scenarios of right-hand yielding actions minus the right-rear overtaking situation of left-hand yielding actions leave 10 basic scenarios, resulting in a total of 90 scenarios. Adding one basic scenario of right-rear overtaking situation leading to left-hand yielding, totaling 10 scenarios, the total number of conflict-type intervention scenarios is 100. For Mode B (Cooperative danger-type intervention): Based on the 11 basic scenarios and two intervention modes, the cooperative danger-type scenarios are combined and similar cooperative redundancies are deducted, resulting in a total of 110 scenarios. The two-target scenarios are summarized. The cumulative conflict-type (100) and cooperative danger-type (110) scenarios generate a total of 210 two-target ship test scenarios.

[0055] To construct a special scenario library, 11 single-target Level 1 hazardous scenarios were designed to simulate actions by the target vessel that violated the rules (including the yielding vessel failing to fulfill its avoidance obligations while maintaining its course and speed, or failing to exercise its right-of-way rights and taking uncoordinated actions). For vessels out of control and with limited maneuverability as described in Article 18 of the Rules, only two test scenarios were required: port crossing and being overtaken, with the target vessel being a trawler and an out-of-control vessel. Under 11 typical situations, Level 3 hazardous emergency scenarios (urgent situations) were conducted, testing the vessel's emergency avoidance capabilities when passing the target vessel's bow or stern. A total of 35 special test scenarios were generated from these scenarios.

[0056] This step constructed a three-level complete scenario library covering single-target scenario libraries (33), two-target scenario libraries (210), and special scenario libraries (35), totaling 278 test scenarios. For the first time, it established quantitative reference answers for 8 secondary indicators for each scenario, forming a quantitative mapping system between scenarios, evaluation indicators, and reference answers. This achieved the completeness of scenario coverage and the traceability of evaluation standards for algorithm completeness testing.

[0057] S3. Based on scenario-based scoring rules and a completeness test scenario library, a ship automatic collision avoidance simulation test platform is established. The ship automatic collision avoidance simulation test platform is used to conduct simulation tests on each scenario in the completeness test scenario library to obtain the first test report. Specifically, step S3 further includes: establishing a ship automatic collision avoidance simulation test platform based on a ship maneuvering simulator, and integrating the VICAD algorithm under test and its automatic evaluation software module into the platform in the form of a dynamic library; Based on the pre-set specific system and rule requirements, determine the secondary indicator quantification standards for each scenario in the completeness test scenario library; The evaluation module compares the actual performance data of the algorithm under test in a certain scenario with the pre-stored quantitative reference answers for each secondary indicator of the scenario. Based on the preset scoring rules and membership functions, it automatically calculates the score of each secondary indicator and obtains the overall evaluation score of the scenario based on the analytic hierarchy process. Any violations or dangerous events are recorded. Repeat the above steps until all selected scenarios have been tested. Summarize all test results and generate a structured first test report.

[0058] In this embodiment, this step aims to construct an automated testing platform integrating a simulation environment, algorithm interface, evaluation engine, and reporting system to achieve closed-loop, efficient, and objective testing of collision avoidance algorithms. Specifically, the ship automatic collision avoidance simulation testing platform, based on a ship maneuvering simulator, will test the VICAD algorithm and its automatic evaluation software module (such as...). Figure 9 As shown, it is integrated into the platform (e.g., as a dynamic library) in the form of a dynamic library. Figure 1(As shown); Next, based on the specific system and rule requirements, quantitative standards are determined for the secondary indicators (encounter situation, danger assessment threshold, danger level, avoidance action method, action timing, action range and number of actions, and track deviation) of each scenario for the complete scenario library designed in step two.

[0059] The evaluation module compares the actual performance data of the algorithm under test in the scenario (such as the timing and magnitude of actual actions, final DCPA, etc.) with the pre-stored "standard answers" for the scenario item by item. Based on preset scoring rules and membership functions, such as... and The system establishes scoring rules and membership functions for the collision safety distance for yielding vessels and the overtaking maneuver, automatically calculating scores for each secondary indicator (such as timing, compliance, and economy). Based on the analytic hierarchy process (AHP), an overall evaluation score for the scenario is obtained, and any violations or dangerous events are recorded. After testing all selected scenarios, the test results for all scenarios are summarized to generate a structured and detailed test report.

[0060] In this embodiment, Figure 9 The VICAD algorithm is integrated using the PIDVCA algorithm as an example. The PIDVCA algorithm's completeness is tested and optimized using a scenario library. The PIDVCA algorithm is tested one by one based on the complete scenario library, and any vulnerabilities discovered during the testing process are optimized to ensure that subsequent tests are successful until all scenario libraries are fully tested.

[0061] S4. Obtain historical AIS data of the actual ship in the selected waters, and use the ship automatic collision avoidance simulation test platform to conduct simulation tests on the historical AIS data of the actual ship to obtain the second test report. Specifically, step S4 further includes: acquiring historical AIS data of real ships in the selected waters, cleaning, filtering and interpolating the raw data to eliminate outliers and missing values, and reconstructing the complete and continuous spatiotemporal trajectories of multiple ships in the waters within a specific time period based on the processed data, forming a real traffic flow basic scenario library that can be called by the simulation system; Typical navigation segments with interaction between the ship and the target ship are selected from the real traffic flow basic scenario library. In the ship automatic collision avoidance simulation test platform, the ship's historical trajectory is used as the reference baseline, but its control is transferred to the algorithm under test. The target ship runs strictly according to its historical trajectory, thus constructing a dynamic test scenario that includes actual uncertainty and complexity. The comprehensive performance of the tested algorithm in an open and real environment was evaluated using a ship automatic collision avoidance simulation test platform. A second test report was generated, which included the adaptability to irregular or ambiguous behavior of the target ship, the ability to identify dangers and prioritize decisions in complex encounters with multiple targets, the overall economy of long-term continuous collision avoidance decisions, and whether the decision-making behavior conforms to good seamanship and is interpretable.

[0062] In this embodiment, the present invention aims to test the completeness of the algorithm. To fully verify the completeness of the aforementioned algorithm completeness test scenario library, algorithm verification using real navigation data (real AIS historical data) is required. When encounter scenarios involving collisions or insufficient safety margins occur during the test process, these encounter scenarios are added to the corresponding scenario library to improve the completeness test scenario library. Specifically, real ship AIS historical data from selected waters is imported, and the raw data is cleaned, filtered, and interpolated to eliminate outliers and missing values. Based on the processed data, the complete and continuous spatiotemporal trajectories of multiple ships in the waters within a specific time period are reconstructed, forming a basic scenario library of real traffic flow that can be called by the simulation system. From the reconstructed real traffic flow, typical navigation segments with interaction between the ship and the target ship are selected. In the simulation platform, the ship's historical trajectory is used as a reference baseline, but its control is transferred to the collision avoidance algorithm under test. The target ship then strictly follows its historical trajectory, thereby constructing a highly realistic dynamic test scenario that includes actual uncertainties and complexities. The system focuses on evaluating the algorithm's overall performance in open, real-world environments, including: its adaptability to irregular or ambiguous behavior of target vessels, its ability to identify hazards and prioritize decisions in complex multi-target encounters, the overall economic efficiency of long-term continuous avoidance decisions, and whether the decision-making behavior conforms to good seamanship and is interpretable.

[0063] S5. Based on the first test report and the second test report, perform attribution analysis and clustering on the failure scenarios that occurred during the test, automatically generate targeted enhancement test scenarios and supplement them to the completeness test scenario library, and generate a multi-dimensional quantitative completeness assessment report. Specifically, step S5 further includes: according to the first test report and the second test report, all scenarios that meet the failure judgment conditions are marked as failure scenarios, and the complete feature vector of the failure scenario is extracted. The failure judgment conditions include any one of collision occurrence and safety margin being lower than a preset safety threshold. The feature vector includes at least two of the following: basic encounter features, danger level features, avoidance action features, failure mode features, and target ship behavior features. Based on the complete feature vector of the failure scenario, failure attribution analysis is performed to determine the cause of failure and the category of the scenario, generate attribution analysis results, and automatically classify and add the failure scenario to the corresponding scenario library according to the attribution analysis results. Quantitative reference answers are generated for newly added failure scenarios. The cause of failure includes at least one of the following: too late timing, insufficient magnitude, incorrect judgment, target incoordination, and failure of multi-target coordination. Cluster analysis is performed on multiple failure scenarios within a preset time period to identify high-frequency failure mode clusters. The occurrence frequency of each failure mode cluster is counted, and failure mode clusters with occurrence frequencies exceeding the weakness threshold are identified as weak links. The cluster analysis is based on the failure mode feature space, which includes at least two dimensions from the encounter situation type, target ship behavior type, hazard level, and failure cause. To address the weak links, based on the common characteristics of failure modes, the variation dimensions are selected and combined using Cartesian products to generate multiple targeted enhancement test scenarios. The targeted enhancement scenarios are then prioritized based on a risk assessment model. The variation dimensions include at least one of the following: target ship maneuvering characteristic parameters, encounter geometry parameters, hazard level parameters, and timing parameters. Conduct a multi-dimensional quantitative completeness assessment and generate a completeness report. The completeness report includes at least two of the following: scenario scale indicators, scenario source composition, feature space coverage, coverage of international maritime collision avoidance rules provisions, and degree of reinforcement of weak links.

[0064] In this embodiment, during the testing of the algorithm under test, the algorithm's performance is monitored in real time. When the failure judgment condition is met, it is automatically marked as a failure scenario and its feature vector is extracted. Attribution analysis is performed based on the feature vector to determine the cause of failure and the category of the scenario. The failure scenario is automatically classified and added to the corresponding scenario library. Cluster analysis is performed on multiple failure scenarios to identify high-frequency failure mode clusters. Failure mode clusters with an occurrence frequency exceeding the weakness threshold are identified as weak links. For weak links, multiple targeted enhancement test scenarios are generated. Version numbers and change logs for the scenario library are established, and a multi-dimensional quantitative completeness report is generated. The failure judgment condition includes any one of the following: a collision occurs, or the nearest encounter distance DCPA is less than the collision critical distance DA. col The safety margin is lower than a preset safety threshold; the causes of failure include at least one of the following: late timing, insufficient magnitude, misjudgment, target incoordination, and failure of multi-target coordination. The feature vector includes at least two of the following: basic encounter features, hazard level features, avoidance action features, failure mode features, and target vessel behavior features; the basic encounter features include encounter situation type, ship size, target vessel size, relative bearing, relative distance, DCPA, and TCPA; the failure mode features include failure type and failure cause. Cluster analysis is performed based on the failure mode feature space, which includes at least two dimensions of encounter situation type, target vessel behavior type, hazard level, and failure cause. Generating targeted augmentation test scenarios includes: for the identified weak points, based on the common features of failure modes, selecting variation dimensions and performing Cartesian product combinations to generate multiple targeted augmentation test scenarios; the variation dimensions include at least one of the following: target vessel maneuvering characteristic parameters, encounter geometric parameters, hazard level parameters, and timing parameters.

[0065] Specifically, during the testing of the algorithm under test, the algorithm's performance is monitored in real time. When the failure judgment conditions are met, it is automatically marked as a failure scenario, and the complete feature vector of the failure scenario is extracted. Based on the feature vector of the failure scenario, failure attribution analysis is performed to determine the cause of failure and the category of the scenario. Multiple failure scenarios within a preset time period are clustered to identify high-frequency failure mode clusters. The occurrence frequency of each failure mode cluster is counted, and failure mode clusters with an occurrence frequency exceeding the weakness threshold are identified as weak links. For the identified weak links, based on the common features of the failure modes, the variation dimension is selected for Cartesian product combination to generate multiple targeted enhancement test scenarios. A multi-dimensional quantitative completeness assessment is generated.

[0066] S6. For the supplemented completeness test scenario library, continue to use the ship automatic collision avoidance simulation test platform for simulation testing, and repeat the evaluation until all scenario libraries pass the test.

[0067] Specifically, in this embodiment, to further verify the characteristics of this method, based on the constructed VICAD algorithm simulation test and automatic evaluation system, the VICAD algorithm uses the PIDVCA algorithm as an example, selecting the following experimental ship types: 3127 is a 135m long frigate with a full speed of 30 knots and a half speed of 22 knots; 3040 is a 198m long roll-on / roll-off ship with a full speed of 19.86 knots and a half speed of 9.3 knots; 3220 is a 346.8m long very large crude carrier with a full speed of 16.47 knots. The PIDVCA algorithm is simulated and automatically evaluated using a multi-ship encounter scenario, mainly from the perspectives of compliance, safety, and economic efficiency. A multi-intelligent ship simulation test mode is used to verify the coordination among multiple intelligent ships implementing avoidance actions based on the collision avoidance decision determined by the key ship. Figure 10 and Figure 11 As shown, this is a test scenario of dual-intelligent and fully intelligent ships encountering three simulated ships (3127, 3040 and 3220 respectively, in a counterclockwise direction with a heading of 0°) that are in Level 1 danger due to the yielding ship missing the best time to avoid a collision.

[0068] in, Figure 10 The top left corner shows the initial situation with relative vectors, indicating a risk of collision between the two target ships. The smart ship with a heading of 0° should give way to both target ships, with the priority ship to avoid being the target ship crossing to starboard. The smart ship with a heading of 180° is both a straight-ahead ship and a ship giving way to the other; since the initial situation has reached a point where immediate course correction is necessary, both smart ships are designated as the priority ship to avoid. Figure 6The established models obtained reversal ranges of 26° and 42° respectively. The test results showed that, since both yielding vessels took timely and significant reversals to avoid each other, although the non-intelligent vessel failed to take evasive action against the approaching vessel on its right as required by the Rules, as it was the key vessel for the two intelligent vessels to avoid, all indicators of the two intelligent vessels met the requirements throughout the entire process of avoiding it.

[0069] Figure 11 The initial encounter situation and Figure 10 The differences are the same: all three vessels are intelligent ships and are in a Level 1 danger state. The course-changing ranges for intelligent ships with headings of 0° and 180° are the same as above. Figure 10 After the non-intelligent vessel was converted to an intelligent vessel, it was also in a Level 1 danger with the other two vessels. The course change was 30°. It immediately took a large course change to the right to avoid the other vessel. The evaluation results showed that because all three vessels took action in accordance with the "Rules" to avoid the key vessel, the safety margin between them was larger than expected. Therefore, the two intelligent vessels that encountered each other resumed their course ahead of time, and the course deviation was significantly smaller than the reference answer.

[0070] In summary, the system and method provided by this invention achieve dynamic quantification and adaptive adjustment of evaluation standards at the technical level, construct a comprehensive test scenario library covering rules, geometric features, and edge cases, and form a full-link automated evaluation system from scenario design, automatic testing, real data verification to closed-loop iteration of the scenario library. At the application level, this invention not only provides standardized testing tools and quantitative evaluation criteria for algorithm development, but also ensures that the algorithm fits actual maritime scenarios through real AIS verification, significantly improving the reliability and practicality of intelligent collision avoidance systems. At the industry level, this invention can lower the threshold for algorithm development and testing, support the compliance certification and technological achievement transformation of intelligent ships, and has significant practical implications for promoting the large-scale application of intelligent collision avoidance technology for ships.

[0071] Please see Figure 12 The second embodiment of the present invention provides a dynamic quantification completeness testing device for ship intelligent collision avoidance algorithms, comprising: The rule construction unit 101 is used to construct dynamic quantitative scenario-based scoring rules. The scenario-based scoring rules include multi-level indicators in three dimensions: safety, compliance and economy. The evaluation threshold of each indicator can be dynamically adjusted according to the ship size, encounter situation and hazard level. The scenario library construction unit 102 is used to construct a completeness test scenario library, which includes a single-target ship scenario library, a two-target ship scenario library, and a special scenario library, and presets quantitative reference answers for each secondary indicator for each scenario library; The first test unit 103 is used to establish a ship automatic collision avoidance simulation test platform based on scenario-based scoring rules and a complete test scenario library, and to use the ship automatic collision avoidance simulation test platform to perform simulation tests on each scenario in the complete test scenario library to obtain the first test report. The second test unit 104 is used to acquire historical AIS data of a real ship in the selected waters, and to conduct simulation tests on the historical AIS data of the real ship using the ship automatic collision avoidance simulation test platform to obtain a second test report. The optimization unit 105 is used to perform attribution analysis and clustering of failure scenarios that occur during the test based on the first test report and the second test report, automatically generate targeted enhancement test scenarios and supplement them to the completeness test scenario library, and generate a multi-dimensional quantitative completeness evaluation report. Iteration unit 106 is used to continue to conduct simulation tests using the ship automatic collision avoidance simulation test platform for the supplemented completeness test scenario library, and repeat the evaluation until all scenario libraries are tested and qualified.

[0072] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for dynamic quantitative completeness testing of intelligent collision avoidance algorithms for ships, characterized in that, include: A dynamic and quantitative scenario-based scoring rule is constructed, which includes multi-level indicators in three dimensions: safety, compliance, and economy. The evaluation thresholds for each indicator can be dynamically adjusted according to the ship's size, encounter situation, and hazard level. A comprehensive test scenario library is constructed, which includes a single-target ship scenario library, a two-target ship scenario library, and a special scenario library, and quantitative reference answers for each secondary indicator are preset for each scenario library; A ship automatic collision avoidance simulation test platform was established based on scenario-based scoring rules and a complete test scenario library. The ship automatic collision avoidance simulation test platform was used to conduct simulation tests on each scenario in the complete test scenario library to obtain the first test report. Obtain historical AIS data of actual ships in the selected waters, and use the ship automatic collision avoidance simulation test platform to conduct simulation tests on the historical AIS data of actual ships to obtain a second test report; Based on the first and second test reports, attribution analysis and clustering are performed on the failure scenarios that occurred during the test, and targeted enhancement test scenarios are automatically generated and added to the completeness test scenario library. At the same time, a multi-dimensional quantitative completeness assessment report is generated. For the supplemented and complete test scenario library, continue to use the ship automatic collision avoidance simulation test platform for simulation testing, and repeat the evaluation until all scenarios in the library pass the test.

2. The method for dynamic quantitative completeness testing of ship intelligent collision avoidance algorithms according to claim 1, characterized in that, A dynamic, quantitative, scenario-based scoring rule is constructed. This rule includes multi-level indicators across three dimensions: safety, compliance, and economy. The evaluation thresholds for each indicator can be dynamically adjusted based on ship size, encounter situation, and hazard level. Specifically: Based on the preset encounter information, a ship encounter situation and attribute identification model is constructed, including the encounter situation with the relative bearing of the approaching ship, the encounter attributes, and the division of avoidance responsibilities; The hazard assessment is based on the ideal safe encounter distance, the critical safe encounter distance, and the critical collision distance. Based on the geometric analysis of the relative motion of the ships, a quantitative model of the threshold for each hazard level is established. The critical safe encounter distance and the critical collision distance are dynamically calculated based on the lengths of the ship and the target ship, the angle between the headings of the two ships, and the turning performance parameters of the ship. Based on the collision avoidance decision parameters, a collision avoidance parameter quantification model is constructed. The collision avoidance decision parameters include the optimal timing for taking collision avoidance action, the magnitude of the action, and the quantification method for predicting the rerouting point timing. The optimal timing and the magnitude of the action are dynamically calculated based on the current relative distance, relative speed, the turning performance of the ship itself, and the motion parameters of the target ship. Based on the ship encounter situation and attribute identification model, the threshold quantification model for each hazard level, and the collision avoidance parameter quantification model, the analytic hierarchy process is used to generate scenario-based scoring rules. The threshold values ​​of each indicator used in the scenario-based scoring rules are dynamically adjusted according to the ship size and hazard level. The scenario-based scoring rules include three primary indicators and eight secondary indicators. The primary indicators are the criteria, and the secondary indicators are the indicators. The three primary indicators are safety, compliance, and economy. The eight secondary indicators are: two secondary indicators under safety, namely collision hazard assessment threshold and hazard level classification; five secondary indicators under compliance, namely encounter situation and liability judgment, avoidance action method and direction, avoidance action timing, avoidance action range, and avoidance action frequency; and one secondary indicator under economy, namely flight deviation.

3. The method for dynamic quantitative completeness testing of ship intelligent collision avoidance algorithm according to claim 2, characterized in that, The construction of a single-target ship scenario library is as follows: The ship type, course, and speed are fixed, the target ship is set as an intelligent ship that complies with the International Regulations for Preventing Collisions at Sea, and 11 typical basic ship encounter situations are determined according to the International Regulations for Preventing Collisions at Sea. For each basic ship encounter situation, based on 2 hazard levels, 3 test scenarios are designed. The 11 typical basic ship encounter situations are multiplied by the 3 test scenarios respectively to obtain 33 single-objective test scenarios. These are combined to generate a single-objective test scenario library, and quantitative reference answers for each secondary indicator are preset for the single-objective test scenario library. The three test scenarios are: Level 1 danger (test scenario where the best opportunity has not been missed), Level 1 danger (test scenario where the best opportunity has been missed), and Level 2 danger (test scenario).

4. The method for dynamic quantitative completeness testing of ship intelligent collision avoidance algorithm according to claim 1, characterized in that, Construct a two-target ship scenario library and a special scenario library, specifically as follows: Eleven single-target ship scenarios with Level 1 danger and no missed optimal opportunity are used as basic seed scenarios. A third target ship is introduced using conflict-type intervention mode and collaborative danger-type intervention mode respectively, generating 210 two-target ship test scenarios. A two-target test scenario library is generated, and quantitative reference answers for each secondary indicator are preset for the two-target test scenario library. There are 100 conflict-type intervention scenarios and 110 collaborative danger-type intervention scenarios. Among them, the conflict-type intervention mode means that a third vessel appears on the avoidance path planned by this vessel to avoid the first target vessel, forming a new conflict. The collaborative danger-type intervention mode means that the third vessel and the first target vessel together pose a collision hazard to this vessel, forming a multiple threat. In 11 single-target vessel scenarios where the first level of danger has not been missed and the best opportunity has not been missed, the target vessel is set to take actions that do not comply with the rules, resulting in 11 corresponding special target vessel scenarios. Actions that do not comply with the rules include giving way vessels failing to fulfill their avoidance obligations and maintaining their course and speed or failing to exercise the right of straight-going vessels and taking uncoordinated actions. In the scenario of a left-hand cross encounter, the target vessel is set as a trawler, and in the scenario of being overtaken, the target vessel is set as an out-of-control vessel, thus creating two special target vessel scenarios. Under 11 typical ship encounter situations, two scenarios were set up: the ship passing by the bow of the target ship and the ship passing by the stern of the target ship. A total of 22 corresponding special target ship scenarios were generated to test the emergency avoidance capability of the algorithm. All the above-mentioned special target ship scenarios are summarized to generate a special scenario library, and quantitative reference answers for each secondary indicator are preset for the special scenario library.

5. The method for dynamic quantitative completeness testing of ship intelligent collision avoidance algorithm according to claim 1, characterized in that, A ship automatic collision avoidance simulation test platform was established based on scenario-based scoring rules and a completeness test scenario library. The platform was used to conduct simulation tests on each scenario in the completeness test scenario library, resulting in a first test report, as follows: A ship automatic collision avoidance simulation test platform based on a ship maneuvering simulator was established, and the VICAD algorithm under test and its automatic evaluation software module were integrated into the platform in the form of a dynamic library. Based on the pre-set specific system and rule requirements, determine the secondary indicator quantification standards for each scenario in the completeness test scenario library; The evaluation module compares the actual performance data of the algorithm under test in a certain scenario with the pre-stored quantitative reference answers for each secondary indicator of the scenario. Based on the preset scoring rules and membership functions, it automatically calculates the score of each secondary indicator and obtains the overall evaluation score of the scenario based on the analytic hierarchy process. Any violations or dangerous events are recorded. Repeat the above steps until all selected scenarios have been tested. Summarize all test results and generate a structured first test report.

6. The method for dynamic quantitative completeness testing of ship intelligent collision avoidance algorithm according to claim 1, characterized in that, Obtain historical AIS data of actual ships in the selected waters, and use a ship automatic collision avoidance simulation test platform to conduct simulation tests on the historical AIS data of actual ships, and obtain a second test report, which is as follows: The system acquires historical AIS data of real ships in the selected waters, cleans, filters and interpolates the raw data to eliminate outliers and missing values, and reconstructs the complete and continuous spatiotemporal trajectories of multiple ships in the waters within a specific time period based on the processed data, forming a basic library of real traffic flow scenarios that can be called by the simulation system. Typical navigation segments with interaction between the ship and the target ship are selected from the real traffic flow basic scenario library. In the ship automatic collision avoidance simulation test platform, the ship's historical trajectory is used as the reference baseline, but its control is transferred to the algorithm under test. The target ship runs strictly according to its historical trajectory, thus constructing a dynamic test scenario that includes actual uncertainty and complexity. The comprehensive performance of the tested algorithm in an open and real environment was evaluated using a ship automatic collision avoidance simulation test platform. A second test report was generated, which included the adaptability to irregular or ambiguous behavior of the target ship, the ability to identify dangers and prioritize decisions in complex encounters with multiple targets, the overall economy of long-term continuous collision avoidance decisions, and whether the decision-making behavior conforms to good seamanship and is interpretable.

7. The method for dynamic quantitative completeness testing of ship intelligent collision avoidance algorithm according to claim 1, characterized in that, Based on the first and second test reports, attribution analysis and clustering are performed on the failure scenarios that occurred during the testing process. Targeted enhancement test scenarios are automatically generated and added to the completeness test scenario library. Simultaneously, a multi-dimensional quantitative completeness assessment report is generated, specifically: According to the first test report and the second test report, all scenarios that meet the failure judgment conditions are marked as failure scenarios, and the complete feature vector of the failure scenario is extracted. The failure judgment conditions include any one of collision occurrence and safety margin being lower than a preset safety threshold. The feature vector includes at least two of the following: basic encounter features, danger level features, avoidance action features, failure mode features, and target ship behavior features. Based on the complete feature vector of the failure scenario, failure attribution analysis is performed to determine the cause of failure and the category of the scenario, generate attribution analysis results, and automatically classify and add the failure scenario to the corresponding scenario library according to the attribution analysis results. Quantitative reference answers are generated for newly added failure scenarios. The cause of failure includes at least one of the following: too late timing, insufficient magnitude, incorrect judgment, target incoordination, and failure of multi-target coordination. Cluster analysis is performed on multiple failure scenarios within a preset time period to identify high-frequency failure mode clusters. The occurrence frequency of each failure mode cluster is counted, and failure mode clusters with occurrence frequencies exceeding the weakness threshold are identified as weak links. The cluster analysis is based on the failure mode feature space, which includes at least two dimensions from the encounter situation type, target ship behavior type, hazard level, and failure cause. To address the weak links, based on the common characteristics of failure modes, the variation dimensions are selected and combined using Cartesian products to generate multiple targeted enhancement test scenarios. The targeted enhancement scenarios are then prioritized based on a risk assessment model. The variation dimensions include at least one of the following: target ship maneuvering characteristic parameters, encounter geometry parameters, hazard level parameters, and timing parameters. Conduct a multi-dimensional quantitative completeness assessment and generate a completeness report. The completeness report includes at least two of the following: scenario scale indicators, scenario source composition, feature space coverage, coverage of international maritime collision avoidance rules provisions, and degree of reinforcement of weak links.

8. A device for testing the dynamic quantitative completeness of intelligent collision avoidance algorithms for ships, characterized in that, include: The rule building unit is used to build dynamic and quantitative scenario-based scoring rules. The scenario-based scoring rules include multi-level indicators in three dimensions: safety, compliance and economy. The evaluation threshold of each indicator can be dynamically adjusted according to the ship size, encounter situation and hazard level. The scenario library construction unit is used to construct a completeness test scenario library, which includes a single-target ship scenario library, a two-target ship scenario library, and a special scenario library, and presets quantitative reference answers for each secondary indicator for each scenario library; The first test unit is used to establish a ship automatic collision avoidance simulation test platform based on scenario-based scoring rules and a complete test scenario library. The ship automatic collision avoidance simulation test platform is used to conduct simulation tests on each scenario in the complete test scenario library to obtain the first test report. The second test unit is used to acquire historical AIS data of actual ships in selected waters, and to conduct simulation tests on the historical AIS data of actual ships using a ship automatic collision avoidance simulation test platform to obtain a second test report. The optimization unit is used to perform attribution analysis and clustering of failure scenarios that occur during the test based on the first test report and the second test report, automatically generate targeted enhancement test scenarios and supplement them to the completeness test scenario library, and generate a multi-dimensional quantitative completeness evaluation report. The iterative unit is used to continue to conduct simulation tests using the ship automatic collision avoidance simulation test platform for the supplemented completeness test scenario library, and to repeatedly evaluate it until all scenario libraries have passed the test.

Images