Recognition and early warning method and system for dangerous behaviors of inland ships

Abstract

The invention provides an identification and early warning method and system for dangerous behaviors of inland ships. The method comprises the following steps: acquiring infrared monitoring video image data of an inland river water area to obtain a standard data set; then, performing ship main shaft identification according to the standard data set, performing bow and stern identification according to a ship gray value, and performing ship motion direction judgment; then, according to a method based on sample generation, a large number of ship target tracking virtual samples are generated; then, inputting the generated ship target tracking virtual sample into a DVIS model tracking training framework to realize tracking training, and obtaining a tracking model; and finally, according to the obtained tracking model and input real sample data, analyzing motion track data and a motion mode of the ship, and carrying out identification and early warning on dangerous behaviors of the ship. According to the method, tracking of ship behaviors under the complex ship shore environment condition is achieved, early warning is conducted on the ship dangerous behaviors, and the recognition rate of the ship navigation dangerous behaviors is further increased.

Description

Technical Field

[0001] The present invention relates to the technical field of ship safety monitoring, and in particular to a method and system for identifying and warning dangerous behaviors of inland river ships. Background Art

[0002] In recent years, with the rapid development of the global shipping industry, inland waterway shipping has continued to develop, the number and types of inland waterway ships have continued to increase, and inland waterway shipping safety is facing more and more challenges. Traditional monitoring methods mainly rely on manual observation and simple image recognition, which cannot timely and accurately identify and warn of dangerous behaviors of ships, which can easily lead to safety accidents.

[0003] Therefore, there is an urgent need for an efficient and reliable inland waterway vessel dangerous behavior identification and early warning solution that can simultaneously meet the monitoring requirements of fast calculation speed and high recognition accuracy, thereby improving the safety of inland waterway shipping. Summary of the invention

[0004] The embodiments of the present invention provide a method and system for identifying and warning dangerous behaviors of inland waterway vessels, which can simultaneously meet the monitoring requirements of fast calculation speed and high identification accuracy, thereby improving the safety of inland waterway navigation.

[0005] To achieve the above object, the embodiments of the present invention adopt the following technical solutions:

[0006] In a first aspect, an embodiment of the present invention provides a method comprising:

[0007] S1. Obtain and preprocess a standard data set, wherein the standard data set includes infrared monitoring video image data of a target inland water area;

[0008] S2. Determine the movement pattern of the ship in the target inland waters;

[0009] S3, generating a ship target tracking virtual sample according to the motion pattern of the ship in the target inland waters and the infrared monitoring video image data, wherein the infrared scene imaging law of the ship is determined by using the continuous infrared monitoring video image data within a specified time period;

[0010] S4, inputting the ship target tracking virtual sample into the DVIS model tracking training framework for tracking training and obtaining a trained tracking model;

[0011] S5. Collect real sample data in real time and input it into the tracking model, and make graded warnings according to the output results of the tracking model, wherein the real sample data collected in real time includes: infrared monitoring video image data taken by monitoring cameras deployed within the river range.

[0012] In a second aspect, an embodiment of the present invention provides a system, comprising:

[0013] A standard data set acquisition module, which acquires and preprocesses a standard data set, wherein the standard data set includes infrared monitoring video image data of a target inland water area;

[0014] The ship movement direction judgment module determines the movement mode of the ship in the target inland waters;

[0015] A virtual sample generation module generates a virtual sample for tracking a ship target based on the motion pattern of the ship in the target inland waters and the infrared surveillance video image data, wherein the infrared scene imaging law of the ship is determined by using the continuous infrared surveillance video image data within a specified time period;

[0016] A virtual sample tracking training module, which inputs the ship target tracking virtual sample into the DVIS model tracking training framework for tracking training and obtains a trained tracking model;

[0017] The ship dangerous behavior identification and warning module collects real sample data in real time and inputs it into the tracking model, and makes graded warnings based on the output results of the tracking model, wherein the real sample data collected in real time includes: infrared monitoring video image data taken by monitoring cameras deployed within the river channel.

[0018] The recognition and early warning method and system for dangerous behaviors of inland ships provided by the embodiment of the present invention obtain infrared monitoring video image data of inland waters, and perform preprocessing such as denoising and enhancement to obtain a standard data set; then, the main axis of the ship is identified according to the standard data set, and the bow and stern are identified according to the gray value of the ship, and the direction of movement of the ship is judged; then, a large number of virtual samples for ship target tracking are generated according to the method based on sample generation; then, the generated virtual samples for ship target tracking are input into the DVIS model tracking training framework to realize tracking training, and a tracking model is obtained; finally, according to the obtained tracking model, the real sample data is input, the motion trajectory data and motion mode of the ship are analyzed, and dangerous behaviors of the ship are identified and early-warned. The method realizes the tracking of ship behaviors under complex ship-shore environment conditions, and early-warns dangerous behaviors of ships, further improving the recognition rate of dangerous behaviors of ship navigation. The infrared ship target tracking method based on sample generation in this scheme can better handle the mutual occlusion and interference between ship targets and other targets, accurately associate and track each target, and can fully consider the characteristics of the ship's motion direction and the imaging law of the port navigation scene to generate virtual samples that fit the reality. This reduces the probability of ship missed detection and misidentification, and improves the accuracy and reliability of overall tracking. It also enables the system to continuously track ship targets in multiple frames, avoiding ID switching and discontinuity during tracking, thereby improving tracking stability and continuity. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A flow chart of a method for identifying and warning dangerous behaviors of inland vessels provided in an embodiment of the present invention.

[0021] Figure 2 A flow chart of a method for determining the direction of ship motion and generating virtual samples provided in an embodiment of the present invention.

[0022] Figure 3 A schematic diagram of PCA ship main shaft identification provided by an embodiment of the present invention.

[0023] Figure 4 A schematic diagram of the imaging law and movement direction law of an infrared ship provided in an embodiment of the present invention.

[0024] Figure 5A flow chart of infrared ship target tracking training and verification based on sample generation provided in an embodiment of the present invention.

[0025] Figure 6 A DVIS flow chart provided by an embodiment of the present invention.

[0026] Figure 7 A framework diagram of a DVIS reference tracker provided in an embodiment of the present invention.

[0027] Figure 8 A flow chart of a dangerous behavior identification and early warning system for inland vessels provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention will be described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention. It can be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "one", "said" and "the" used herein may also include plural forms. It should be further understood that the term "including" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used here may include wireless connection or coupling. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items. It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as generally understood by those of ordinary skill in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0029] A method for identifying and warning dangerous behaviors of inland vessels, such as Figure 1 As shown, including:

[0030] S1. Obtain and preprocess a standard data set, wherein the standard data set includes infrared monitoring video image data of a target inland water area;

[0031] S2. Determine the movement pattern of the ship in the target inland waters;

[0032] S3. Generate a virtual sample for ship target tracking based on the movement pattern of the ship in the target inland waters and the infrared monitoring video image data, wherein the infrared scene imaging law of the ship is determined by using the continuous infrared monitoring video image data within a specified time period; for example: the infrared imaging data is used to reflect the infrared scene imaging law of the ship. The infrared scene imaging law refers to the law that the position of the infrared monitoring causes the image of the ship to get closer or farther away during its operation, such that the image is larger when near and smaller when far. S4. Input the virtual sample for ship target tracking into the DVIS model tracking training framework for tracking training and obtain the trained tracking model;

[0033] S5. Collect real sample data in real time and input it into the tracking model, and make graded warnings according to the output results of the tracking model, wherein the real sample data collected in real time includes: infrared monitoring video image data taken by monitoring cameras deployed within the river range.

[0034] Among them, in the process of acquiring the standard data set, the infrared monitoring video image data of inland waters is obtained, and pre-processing such as denoising and enhancement is performed to obtain the standard data set; in the process of judging the direction of ship movement, according to the standard data set, the principal axis of the ship is identified through principal component analysis, and then the bow and stern are identified according to the gray value of the ship, and the direction of ship movement is judged; in the process of generating virtual samples, according to the characteristics of the ship's movement direction and the imaging law of the port navigation scene, a large number of ship target tracking virtual samples are generated based on the sample generation method; in the process of virtual sample tracking training, the generated ship target tracking virtual samples are input into the DVIS model tracking training framework to realize tracking training, and a tracking model is obtained; in the process of identifying and warning of dangerous ship behaviors, the tracking model is obtained according to the tracking training of the ship target tracking virtual samples, and the real sample data is input to analyze the ship's motion trajectory data and motion pattern, so as to identify and warn of dangerous ship behaviors.

[0035] Identify ships from infrared surveillance video image data and identify the principal axis of the ship in the length direction through PCA;

[0036] According to the identified main axis, the ship is divided into three sections, and the bow position is judged according to the gray value. Among them, the section matching the gray feature of the middle section is the stern;

[0037] The motion mode of the ship is determined using the motion direction of the main axis.

[0038] Among them, the section with similar grayscale features to the middle section is the stern; specifically, Figure 2As shown, we first perform principal component analysis (PCA) to identify the principal axis of the ship. PCA (Principal Component Analysis) is a statistical method that converts a set of variables that may be correlated into a set of linearly uncorrelated variables through orthogonal transformation. The converted set of variables is called principal component. The goal of PCA is to determine the principal component with the largest variance in the data. The larger the variance, the greater the variation in the data along that direction. Therefore, for a ship that is streamlined when viewed from the top, PCA can determine the principal axis in the length direction and divide the ship into three sections, such as Figure 3 shown.

[0039] In the step of judging the direction of movement of the ship, the principal axis of the ship is identified by principal component analysis, and the principal axis in the length direction is determined; the ship is divided into three sections according to the principal axis of the ship, and the bow and stern are judged according to the grayscale value. The section with similar grayscale features to the middle section is the stern, and vice versa is the bow; the principal component with the largest variance in the data is determined. The larger the variance, the greater the change in the data along this direction. The direction of movement of the ship is determined accordingly. This study divides the ship into three sections according to the principal axis. The section with similar grayscale features to the middle section is the stern, and vice versa is the bow. Identifying the bow and stern is key information for tracking and predicting changes in the ship's position. Accurate identification of the direction of movement helps to better track ship targets in subsequent steps.

[0040] The identifying of the principal axis in the length direction of the ship comprises:

[0041] The mask points M corresponding to the ship image are extracted from the infrared surveillance video image data by PCA. n ,y n ), where x1 and y1 represent the horizontal and vertical coordinates of the first vertex of the polygon image, respectively. n ,y n They represent the horizontal and vertical coordinates of the nth vertex of the polygonal image, respectively, where n is a positive integer; det(C-λI)=0, where cov(x,x) represents the covariance between the horizontal coordinates x and x, cov(x,y) represents the covariance between the horizontal coordinates x and y, and so on; I is the unit matrix, λ represents the eigenvalue of C, C represents the covariance matrix; the direction of the eigenvector v1 corresponding to the maximum eigenvalue λ1 is taken as the main axis direction of M, and the main axis direction of M corresponds to the main axis motion direction of the ship. Specifically, the purpose of PCA is to find a vector v=(v x ,v y ), when the point is projected onto this vector, the variance of the point is the largest. This unit vector v represents the main axis direction of the mask point set:

[0042]

[0043] Among them, cov(x,x) represents the covariance between the horizontal coordinates x and x, cov(x,y) represents the covariance between the horizontal coordinates x and y, and so on.

[0044] Then calculate the eigenvalues ​​and eigenvectors of C according to (1):

[0045] det(C-λI)=0 (2)

[0046] Among them, I is the unit matrix, and λ represents the eigenvalue of C. Then the eigenvector v1 corresponding to the maximum eigenvalue λ1 can be obtained, which represents the main direction of M.

[0047] In this embodiment, determining the motion mode of the ship according to the motion direction of the main axis includes: acquiring the area of ​​the ship target area corresponding to the time t1 and the time t2 respectively from the infrared monitoring video image data, wherein the time t1 is earlier than the time t2;

[0048] According to the area of ​​the ship target area corresponding to time t1 and time t2, the ship image that meets the preset feature is screened out, and the preset feature is that the area of ​​the ship target area is larger near and smaller far away; according to the ship image that meets the preset feature and the movement direction of the main axis of the ship, the movement mode of the ship is determined. For example: Further, in Figure 4 In the infrared surveillance video, the ship target appears larger when near and smaller when far. Figure 4 In (a), the ship target moves from time t1 to time t2. The area of ​​the ship target region closer to the infrared camera is larger than the area of ​​the ship target region farther away, which conforms to the characteristics of near larger and far smaller. Figure 4 (b) The ship targets at time t1 and t2 do not meet the characteristics of being larger when near and smaller when far; Figure 4 (c) The ship target moves from time t1 to time t2, and it is judged that the ship is moving downward, which is consistent with the direction of the bow-stern line; Figure 4 (d) It complies with the characteristic that objects are larger when near and smaller when far. The ship target moves from time t1 to time t2, and it is judged that the ship is moving upward, which is inconsistent with the direction of the bow-stern line.

[0049] The method of determining the movement mode of the ship based on the ship image that meets preset characteristics and the movement direction of the main axis of the ship includes: if the error between the area of ​​the ship target area corresponding to time t2 and the area of ​​the ship target area corresponding to time t1 is within a preset range (which can be understood as approximately equal), then it is judged that the ship is in a stationary state; if the movement direction of the main axis of the ship is opposite to the observation direction of the camera, and the area of ​​the ship target area corresponding to time t2 is larger than that at time t1, then it is judged that the ship is moving closer to the camera; if the movement direction of the main axis of the ship is the same as the observation direction of the camera, and the area of ​​the ship target area corresponding to time t2 is smaller than that at time t1, then it is judged that the ship is moving away from the camera.

[0050] For example, based on the change in the position of the ship at time t1 and time t2, the ship's motion mode can be determined as follows: (1) Stationary ship: If the position of the ship at time t1 and time t2 is not much different, and the area is similar, that is, the coordinate position has no obvious change, it can be determined that the ship is in a stationary state; (2) Movement toward the camera: If the movement direction of the ship's position from time t1 to time t2 is opposite to the observation direction of the camera, and the area of ​​the ship at time t1 is smaller than that at time t2, it may indicate that the ship is moving toward the camera; (3) Movement away from the camera: On the contrary, if the movement direction of the ship from time t1 to time t2 is the same as the observation direction of the camera, and the area of ​​the ship at time t1 is larger than that at time t2, it may indicate that the ship is moving away from the camera. Among them, the size of the ship target area is determined according to the infrared ship target recognition mask data. By observing and comparing the position changes of the ship at different times, the ship's motion mode can be more accurately determined, thereby helping to analyze and monitor the ship's motion trajectory.

[0051] In this embodiment, S3 includes: establishing a ship target tracking model, and generating a ship target tracking virtual sample according to the ship target tracking model, wherein the ship target tracking model includes: α represents the parameter of the polynomial, i represents the exponent of the current term, x represents the distance between the ship area and the bottom edge of the image, and n represents the highest power of the polynomial; the area of ​​the ship target area corresponding to time t1 is used as the input of the ship target tracking model, and the output result is used as the area of ​​the ship target area corresponding to the simulated time t2.

[0052] Among them, in the virtual sample generation step, after the bow and stern lines are identified, the image ship position is selected in continuous frames, and according to the characteristics of the ship target in the infrared monitoring video that it is larger when near and smaller when far, the virtual sample generation based on the polynomial curve is performed. Specifically, the virtual sample generation step, that is, the virtual sample generation step: according to the characteristics of the ship's movement direction and the imaging law of the port navigation scene, a large number of ship target tracking virtual samples are generated based on the sample generation method. Specifically, as Figure 2 As shown in the figure, after recognizing the bow-stern line, this study needs to generate virtual samples based on the characteristics of the ship target in the infrared surveillance video, that is, the ship target at time t2 that conforms to the ship's motion direction is generated by the ship target at time t1. It is necessary to calculate the ship position and ship image area at time t2. This study uses the distance between the ship area and the bottom edge of the image as the independent variable and the pixel area of ​​the ship area as the dependent variable to generate a polynomial fitting curve, as shown in formula (3), where α is a parameter.

[0053]

[0054] In this study, the ship at time t1 is scaled to the area calculated by formula (3) and copied to the position t2, thereby generating the ship target tracking sample at time t2. By repeating the above steps, a large number of ship target tracking virtual samples that conform to the imaging law in the infrared camera and the characteristics of the ship target movement direction can be obtained.

[0055] Furthermore, in the virtual sample tracking training step, the generated virtual sample for ship target tracking is input into the DVIS model tracking training framework to implement tracking training, and the ship target is tracked by analyzing the correlation and movement law between adjacent frames, so as to realize unsupervised modeling and tracking of the ship trajectory and obtain a tracking model. Virtual sample tracking training: The generated virtual sample for ship target tracking is input into the DVIS model tracking training framework to implement tracking training and obtain a tracking model. DVIS (Decoupled Video Instance Segmentation Framework) is extended from the image instance segmentation task, and aims to simultaneously segment, detect, and track all instances in the video.

[0056] Specifically, Figure 5 As shown in Figure 1, this study uses the DVIS model tracking training framework to implement tracking training. The DVIS method consists of three independent subtasks: segmentation, tracking, and optimization. The effectiveness of this model relies on two key elements: (1) accurate long-term temporal alignment results are achieved through frame-by-frame association during the tracking process; (2) in the optimization process, the temporal information is effectively utilized based on the aforementioned accurate alignment results. The flowchart of this method and the tracker framework are shown in Figure 1. Figure 5 and Figure 6 shown.

[0057] Furthermore, the association task of objects in adjacent frames is modeled as reconstructing the object query parameters corresponding to the current frame based on the object query parameters of the previous frame, that is, given the query parameters of the object in the previous frame as the reference query parameters, information is aggregated from the target query parameters of the current frame output by the segmenter, and finally the mask and category of the target corresponding to the reference query parameters in the current frame are output. The reference tracker learns the above process by using the reference cross attention mechanism. The reference cross attention mechanism makes full use of the reference query parameters to guide the aggregation of information and blocks the mixing of the reference query parameters and the current information. It is obtained by slightly modifying the standard cross attention mechanism:

[0058] RCA(ID,Q,K,V)=ID+MHA(Q,K,V) (4)

[0059] MHA stands for Multi-Head Attention, where ID, Q, K and V are identification, query, key and value respectively.

[0060] After the reference tracker outputs the target query parameters that are basically aligned in the temporal dimension, it is easy to effectively utilize the temporal features through standard operations such as ID convolution and self-attention mechanism. The designed time refiner aggregates the temporal features by ID convolution and self-attention mechanism. The time refiner optimizes the segmentation results and tracking results based on the temporal features of the object. The reference tracker can be superimposed on any query parameter-based image segmenter to achieve online video segmentation, and the time refiner can also be superimposed on any online video segmenter to obtain more powerful segmentation performance. A spatiotemporal consistency model is established to track the ship target by analyzing the correlation and movement rules of the ship between different frames. This method can utilize the correlation information between adjacent frames to achieve unsupervised modeling and tracking of ship trajectories.

[0061] In this embodiment, the warning according to the output result of the tracking model includes: determining the dangerous behavior of the ship according to the output result of the tracking model, and the types of dangerous behaviors of the ship include: channel deviation, abnormal speed, stopping, abnormal ship direction and close-range meeting, wherein after determining the dangerous behavior of the ship, the risk behavior of the ship is also classified:

[0062] After that, the dangerous behavior of the ship is warned, and the corresponding warning information is generated and sent to the corresponding terminal device. For example: the steps of ship dangerous behavior identification and warning: the tracking model is obtained according to the tracking training of the ship target tracking virtual sample, the real sample data is input, the motion trajectory data and motion pattern of the ship are analyzed, and the dangerous behavior of the ship is identified and warned. Once the dangerous behavior is identified, the system immediately generates warning information and notifies the relevant management personnel through the sound and light alarm device or wireless communication module to take timely measures. For example, in the process of graded warning, the ship risk behavior needs to be graded: low risk, the channel deviates from the predetermined channel by no more than 2m, the ship speed changes within the range of ±5%, and the heading deviates from the predetermined value by no more than 5 degrees; medium risk, the channel deviates from the predetermined channel by more than 2m but not more than 5m, the ship speed changes by more than ±5% but not more than ±10%, and the heading deviates from the predetermined value by more than 5 degrees but not more than 15 degrees; high risk, the channel deviates from the predetermined channel by more than 5m, the ship speed changes by more than ±10%, the heading deviates from the predetermined value by more than 15 degrees, and the distance between the ship and the obstacle or other ships is less than 10m, and there is a risk of immediate collision. When low-risk behavior is detected, the system issues a low-risk warning signal to remind the operator to pay attention but no immediate action is required; when medium-risk behavior is detected, the system issues a medium-risk warning signal and advises the operator to take preventive measures, such as slowing down or adjusting the course; when high-risk behavior is detected, the system issues a high-risk warning signal and automatically initiates emergency measures, such as automatic avoidance or emergency stop. The ship's dangerous behavior identification and warning step, that is, the ship's dangerous behavior identification and warning step 105: According to the tracking training of the ship target tracking virtual sample to obtain the tracking model, the input real sample data, analyze the ship's motion trajectory data and motion pattern, identify and warn the ship's dangerous behavior, once the dangerous behavior is identified, the system immediately generates a warning message, and notifies the relevant management personnel through the sound and light alarm device or the wireless communication module, so that timely measures can be taken. For example: The early warning processing process includes the following steps: information notification, sending early warning information to ship operators and relevant departments through sound and light alarm devices or wireless communication systems, including specific risk levels and recommended response measures; risk ship management and control, enhancing supervision of risk ships, and managing and controlling them when necessary; multi-department coordination, in high-risk situations, the system will notify relevant departments (such as maritime management departments, rescue teams, etc.) and initiate a multi-department coordinated emergency response; real-time recording and playback: the system records all early warnings and processing results for subsequent analysis and improvement.

[0063] In this embodiment, refer to Figure 8This embodiment also provides an inland river ship dangerous behavior identification and early warning system, which corresponds to the above-mentioned inland river ship dangerous behavior identification and early warning method, and can be understood as a system for implementing the above-mentioned method. The system includes a standard data set acquisition module, a ship movement direction judgment module, a virtual sample generation module, a virtual sample tracking training module, a ship dangerous behavior identification and early warning module, and a data storage module connected in sequence, wherein:

[0064] A standard data set acquisition module 201 acquires and preprocesses a standard data set, wherein the standard data set includes infrared monitoring video image data of a target inland water area;

[0065] The ship movement direction determination module 202 determines the movement mode of the ship in the target inland waters;

[0066] The virtual sample generation module 203 generates a ship target tracking virtual sample according to the motion pattern of the ship in the target inland waters and the infrared monitoring video image data, wherein the infrared scene imaging law of the ship is determined by using the continuous infrared monitoring video image data within a specified time period;

[0067] A virtual sample tracking training module 204 inputs the ship target tracking virtual sample into a DVIS model tracking training framework for tracking training and obtains a trained tracking model;

[0068] The ship dangerous behavior identification and warning module 205 collects real sample data in real time and inputs it into the tracking model, and makes graded warnings based on the output results of the tracking model, wherein the real sample data collected in real time includes: infrared monitoring video image data taken by monitoring cameras deployed within the river channel.

[0069] The data storage module 206 is used to store the collected data, processing results and warning information for subsequent analysis and reference.

[0070] The recognition and early warning method and system for dangerous behaviors of inland ships provided by the embodiment of the present invention obtain infrared monitoring video image data of inland waters, and perform preprocessing such as denoising and enhancement to obtain a standard data set; then, the main axis of the ship is identified according to the standard data set, and the bow and stern are identified according to the gray value of the ship, and the direction of movement of the ship is judged; then, a large number of virtual samples for ship target tracking are generated according to the method based on sample generation; then, the generated virtual samples for ship target tracking are input into the DVIS model tracking training framework to realize tracking training, and a tracking model is obtained; finally, according to the obtained tracking model, the real sample data is input, the motion trajectory data and motion mode of the ship are analyzed, and dangerous behaviors of the ship are identified and early-warned. The method realizes the tracking of ship behaviors under complex ship-shore environment conditions, and early-warns dangerous behaviors of ships, further improving the recognition rate of dangerous behaviors of ship navigation. The infrared ship target tracking method based on sample generation in this scheme can better handle the mutual occlusion and interference between ship targets and other targets, accurately associate and track each target, and can fully consider the characteristics of the ship's motion direction and the imaging law of the port navigation scene to generate virtual samples that fit the reality. This reduces the probability of ship missed detection and misidentification, and improves the accuracy and reliability of overall tracking. It also enables the system to continuously track ship targets in multiple frames, avoiding ID switching and discontinuity during tracking, thereby improving tracking stability and continuity.

[0071] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The above is only a specific implementation method of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or replacements that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for identifying and warning dangerous behaviors of inland vessels, characterized in that: include: S1. Obtain and preprocess a standard data set, wherein the standard data set includes infrared monitoring video image data of a target inland water area; S2. Determine the movement pattern of the ship in the target inland waters; S3, generating a ship target tracking virtual sample according to the motion pattern of the ship in the target inland waters and the infrared monitoring video image data, wherein the infrared scene imaging law of the ship is determined by using the continuous infrared monitoring video image data within a specified time period; S4, inputting the ship target tracking virtual sample into the DVIS model tracking training framework for tracking training and obtaining a trained tracking model; S5. Collect real sample data in real time and input it into the tracking model, and make graded warnings according to the output results of the tracking model, wherein the real sample data collected in real time includes: infrared monitoring video image data taken by monitoring cameras deployed within the river range.

2. The method according to claim 1, characterized in that S2 include: Identify ships from infrared surveillance video image data and identify the principal axis of the ship in the length direction through PCA; According to the identified main axis, the ship is divided into three sections, and the bow position is judged according to the gray value. Among them, the section matching the gray feature of the middle section is the stern; The motion mode of the ship is determined using the motion direction of the main axis.

3. The method according to claim 1, characterized in that The identifying of the principal axis in the length direction of the ship comprises: Extracting mask points corresponding to ship images from infrared surveillance video image data through PCA M=(x1,y1)(x2,y2),…,(x n ,y n ), where x1 and y1 represent the horizontal and vertical coordinates of the first vertex of the polygon image, respectively. n ,y n They represent the horizontal and vertical coordinates of the nth vertex of the polygonal image, respectively, where n is a positive integer; det(C-λI)=0, where cov(x,x) represents the covariance between the horizontal coordinate x and the horizontal coordinate x, cov(x,y) represents the covariance between the horizontal coordinate x and the vertical coordinate y, and so on; I is the unit matrix, λ represents the eigenvalue of C, and C represents the covariance matrix; The direction of the eigenvector v1 corresponding to the maximum eigenvalue λ1 is taken as the main axis direction of M, and the main axis motion direction of the ship corresponds to the main axis direction of M.

4. The method according to claim 2, characterized in that Determining the motion mode of the ship according to the motion direction of the main shaft includes: From the infrared monitoring video image data, the area of ​​the ship target area corresponding to the time t1 and the time t2 is obtained, wherein the time t1 is earlier than the time t2; According to the areas of the ship target areas corresponding to time t1 and time t2, respectively, a ship image meeting a preset feature is selected, wherein the preset feature is that the area of ​​the ship target area is larger when it is near and smaller when it is far away; The movement mode of the ship is determined according to the ship image that meets the preset characteristics and the movement direction of the main axis of the ship.

5. The method according to claim 4, characterized in that The determining the motion mode of the ship according to the ship image meeting the preset characteristics and the motion direction of the main axis of the ship comprises: If the error between the area of ​​the ship target area corresponding to time t2 and the area of ​​the ship target area corresponding to time t1 is within a preset range, it is determined that the ship is stationary; If the movement direction of the main axis of the ship is opposite to the observation direction of the camera, and the area of ​​the ship target area corresponding to time t2 is larger than that at time t1, it is judged that the ship is moving closer to the camera; If the movement direction of the main axis of the ship is the same as the observation direction of the camera, and the area of ​​the ship target area corresponding to time t2 is smaller than that at time t1, it is judged that the ship is moving away from the camera.

6. The method according to claim 4, characterized in that S3 includes: A ship target tracking model is established, and a ship target tracking virtual sample is generated according to the ship target tracking model. The ship target tracking model includes: α represents the parameter of the polynomial, i represents the exponent of the current term, x represents the distance between the ship area and the bottom edge of the image, and n represents the highest power of the polynomial; The area of ​​the ship target region corresponding to time t1 is used as the input of the ship target tracking model, and the output result is used as the area of ​​the ship target region corresponding to the simulated time t2.

7. The method according to claim 1, characterized in that In S5, the issuing of an early warning according to the output result of the tracking model includes: According to the output result of the tracking model, the dangerous behavior of the ship is determined, and the types of dangerous behaviors of the ship include: channel deviation, abnormal speed, stopping, abnormal ship direction and close-range meeting. After the dangerous behavior of the ship is determined, the risk behavior of the ship is also classified: Afterwards, an early warning is issued for the dangerous behavior of the ship, and the corresponding early warning information is generated and sent to the corresponding terminal device.

8. The method according to claim 7, characterized in that The classification of ship risk behaviors includes: The criteria for determining the low risk level include: the channel does not deviate from the planned channel by more than 2m, the ship speed changes within the range of ±5% and the heading deviates from the planned value by no more than 5 degrees; The criteria for determining the medium risk level include: the channel deviates from the planned channel by more than 2m but not more than 5m, the ship speed changes by more than ±5% but not more than ±10%, and the course deviates from the planned value by more than 5 degrees but not more than 15 degrees; The criteria for determining a high risk level include: the channel deviates from the planned channel by more than 5m, the ship speed changes by more than ±10%, the heading deviates from the planned value by more than 15 degrees, and the distance between the ship and obstacles or other ships is less than 10m.

9. The method according to claim 7 or 8, characterized in that: Also includes When the dangerous behavior of the ship is detected to be at a low risk level, the system will issue a low risk warning signal to prompt the operator to pay attention but no immediate action is required; when the dangerous behavior of the ship is detected to be at a medium risk level, the system will issue a medium risk warning signal and suggest the operator to take preventive measures, such as slowing down or adjusting the course; when the dangerous behavior of the ship is detected to be at a high risk level, the system will issue a high risk warning signal and automatically initiate emergency measures, such as automatic avoidance or emergency stop.

10. An identification and early warning system for dangerous behaviors of inland vessels, characterized in that: include: A standard data set acquisition module, which acquires and preprocesses a standard data set, wherein the standard data set includes infrared monitoring video image data of a target inland water area; The ship movement direction judgment module determines the movement mode of the ship in the target inland waters; A virtual sample generation module generates a virtual sample for tracking a ship target based on the motion pattern of the ship in the target inland waters and the infrared surveillance video image data, wherein the infrared scene imaging law of the ship is determined by using the continuous infrared surveillance video image data within a specified time period; A virtual sample tracking training module, which inputs the ship target tracking virtual sample into the DVIS model tracking training framework for tracking training and obtains a trained tracking model; The ship dangerous behavior identification and warning module collects real sample data in real time and inputs it into the tracking model, and makes graded warnings based on the output results of the tracking model, wherein the real sample data collected in real time includes: infrared monitoring video image data taken by monitoring cameras deployed within the river channel.

Images