River ship violation identification method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of violation recognition, in particular to a river channel ship violation recognition method and device, electronic equipment, a storage medium and an application server, according to the river channel ship violation recognition method, feature extraction precision is enhanced through an improved YOLOv8 model by adopting a Swin Transform module, multi-scale feature fusion is achieved in combination with a parallel MaxPool and CSPN structure, and the recognition accuracy of river channel ship violation is improved. Stable target tracking is realized by combining an SORT or DeepSORT algorithm and a Hungary algorithm, and illegal parking, border crossing and other illegal behaviors are accurately judged by combining an identification frame and actual distance mapping estimation speed, so that precision, automation and intelligentization of a supervision process are realized, the pain points of low efficiency and insufficient coverage of traditional manual supervision are effectively solved, and the supervision efficiency is improved. And efficient and reliable technical support is provided for intelligent river management.

Description

Technical Field

[0001] The present invention relates to the technical field of violation identification, and in particular to a method, device, electronic equipment, and storage medium for river vessel violation identification. Background Art

[0002] In recent years, with the rapid development of inland waterway shipping, the number of river vessels has increased dramatically, and the demand for real-time monitoring of vessels has become increasingly urgent. Traditional means of monitoring river vessels mainly rely on manual patrols, fixed camera monitoring or radar systems. However, manual patrols have the problems of low efficiency, high cost and limited coverage; fixed cameras are easily affected by weather (such as rain and fog, insufficient light at night) and river terrain, resulting in monitoring blind spots; although the radar system can work all day, the equipment deployment and maintenance costs are high, and it is difficult to accurately identify small vessels in complex river environments. In addition, existing technologies mostly use a single data source and lack multimodal data fusion capabilities, resulting in a high false alarm rate and insufficient real-time performance. Therefore, there is an urgent need for a low-cost, highly adaptable solution that can accurately monitor river vessels in real time. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, device, electronic equipment and storage medium for identifying violations of river vessels.

[0004] In a first aspect, an embodiment of the present invention provides a method for identifying violations by river vessels, which is applied to a server. The method includes:

[0005] Obtaining the image frame to be recognized;

[0006] The improved YOLOv8 model is used to detect targets in image frames, identify and locate ships, and obtain target recognition results.

[0007] Based on the target recognition results, the SORT or DeepSORT algorithm is used to continuously track the ship. The Hungarian algorithm is used to associate the detected target ship with the targets in the tracking list, and the motion trajectory and motion parameters are updated.

[0008] If the target ship's motion parameters, motion trajectory, and set rules determine that the ship has violated the rules, a warning message indicating the violation will be returned.

[0009] Among them, the backbone network of the improved YOLOv8 model uses the SwinTransformer module to replace the original C2f module, and the convolutional layer of the backbone network also includes multiple parallel MaxPool operations combined with the CSPN structure.

[0010] In combination with the first aspect, the step of obtaining the image frame to be identified includes:

[0011] In response to a video stream acquisition instruction, acquiring an IP address carried in the instruction;

[0012] Establishing a communication connection between the control unit and the camera based on the IP address; the camera corresponds to the IP address;

[0013] Receive the video stream captured by the camera and use the deepstream method to accelerate the video stream;

[0014] Frame capture is performed on the accelerated video stream to obtain a plurality of continuous image frames.

[0015] Combined with the first aspect, the improved YOLOv8 model includes:

[0016] The backbone network uses the Swin Transformer module to replace the original C2f module, and adds multiple MaxPool operations in parallel in the convolutional layer, combined with the CSPN structure to achieve multi-scale feature extraction;

[0017] Feature fusion network, using PANet structure for multi-level feature fusion;

[0018] The detection head adopts a decoupled head structure to separate the classification and regression tasks.

[0019] Combined with the first aspect, the improved YOLOv8 model also includes:

[0020] The SPP layer is added to the backbone network to convert feature maps of any size into feature vectors of fixed size to avoid image scaling distortion.

[0021] Combined with the first aspect, based on the target recognition results, the SORT or DeepSORT algorithm is used to continuously track the ship, the Hungarian algorithm is used to associate the detected target with the targets in the tracking list, and the motion trajectory and motion parameters are updated, including the following steps:

[0022] For each image frame, the target vessel position is detected using the SORT or DeepSORT algorithm based on the target recognition results, and a detection frame containing the position information is generated;

[0023] The Hungarian algorithm is used to match the detection frame of the current frame with the predicted position of the target in the tracking list, and the similarity between the detection frame and the tracked target is calculated to obtain the matching result;

[0024] Associating the detection target with the tracking target based on the matching results;

[0025] For the successfully associated tracking target, use Kalman filtering to predict its motion state in the next frame and update the position and velocity information in the tracking list;

[0026] For unassociated new detection targets, initialize a new tracker and add it to the tracking list;

[0027] Continuously output the motion trajectory of each target in the tracking list and mark the target ID to achieve continuous tracking across frames to update the motion trajectory.

[0028] In conjunction with the first aspect, the server is connected to the management platform for communication;

[0029] If it is determined based on the target vessel's motion parameters, motion trajectory, and set rules that the vessel has violated a rule, before returning a prompt message indicating the violation, the method further includes:

[0030] Send a preset rule acquisition request to the management platform, which carries the image frame, target recognition results, updated motion trajectory and motion parameters;

[0031] Receive the preset rules returned by the management platform.

[0032] In conjunction with the first aspect, if it is determined based on the target vessel's motion parameters, motion trajectory, and set rules that the vessel has violated a rule, after returning a prompt message indicating the violation, the method further includes:

[0033] Collect and compile violation data, categorizing them by violation period, monitoring area, and vessel type;

[0034] Conduct correlation analysis between the frequency of violation types and the monitoring area, and generate a violation distribution map that shows the temporal and spatial distribution of violations and their correlation with vessel types;

[0035] Based on the distribution map and correlation analysis results, targeted management strategies are output, including optimizing patrol periods, adjusting key monitoring areas, or restricting navigation rights for specific vessel types.

[0036] In a second aspect, an embodiment of the present application further provides a device for identifying violations of river vessels, the device comprising:

[0037] An acquisition module, used for acquiring an image frame to be identified;

[0038] The recognition module is used to detect targets in image frames using the improved YOLOv8 model, identify and locate ships, and obtain target recognition results;

[0039] The tracking module is used to continuously track the ship based on the target recognition results using the SORT or DeepSORT algorithm, associate the detected target ship with the targets in the tracking list using the Hungarian algorithm, and update the motion trajectory and motion parameters;

[0040] A violation prompt module is used to return a violation prompt message if it is determined that a ship has violated the rules based on the target ship's motion parameters, motion trajectory and set rules;

[0041] Among them, the backbone network of the improved YOLOv8 model uses the SwinTransformer module to replace the original C2f module, and the convolutional layer of the backbone network also includes multiple parallel MaxPool operations combined with the CSPN structure.

[0042] In a third aspect, the present application provides an electronic device, which includes a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above method.

[0043] In a fourth aspect, the present application provides a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the above-mentioned method is executed.

[0044] The embodiments of the present invention bring the following beneficial effects: the method, device, electronic device, storage medium, and application server provided by the present application for identifying river vessel violations, the method comprising: obtaining an image frame to be identified; using an improved YOLOv8 model to perform target detection on the image frame, identifying and locating the vessel, and obtaining a target recognition result; based on the target recognition result, using a SORT or DeepSORT algorithm to continuously track the vessel, and using the Hungarian algorithm to associate the detected target vessel with the target in the tracking list, and updating the motion trajectory and motion parameters; if it is determined that the vessel has violated the rules based on the motion parameters, motion trajectory, and set rules of the target vessel, a prompt message of the violation is returned; wherein, the backbone network of the improved YOLOv8 model uses a SwinTransformer module to replace the original C2f module, and the convolutional layer of the backbone network also includes multiple parallel MaxPool operations combined with the CSPN structure.

[0045] The river vessel violation identification method provided in this application uses an improved YOLOv8 model and a SwinTransformer module to enhance feature extraction accuracy, combines parallel MaxPool and CSPN structures to achieve multi-scale feature fusion, and improves the accuracy of vessel monitoring. It then combines the SORT or DeepSORT algorithm with the Hungarian algorithm to achieve stable target tracking, combines the recognition box with the actual distance mapping to estimate the speed, and accurately determines violations such as illegal parking and crossing the boundary, thereby realizing the precision, automation and intelligence of the supervision process, effectively solving the pain points of low efficiency and insufficient coverage of traditional manual supervision, and providing efficient and reliable technical support for smart river management.

[0046] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or understood by practicing the present invention. The purposes and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0047] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 A method for identifying river vessel violations provided by an embodiment of the present invention;

[0050] Figure 2 A device for identifying river vessel violations provided by an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of the architecture of the YOLOv8 model in related technology;

[0053] Figure 5 A schematic diagram of the architecture of the improved YOLOv8 model provided in an embodiment of the present invention;

[0054] Figure 6 A schematic diagram of the architecture of the SPP layer in the improved YOLOv8 model provided in an embodiment of the present invention;

[0055] Figure 7 A schematic diagram of the architecture of the Swin Transformer module in the improved YOLOv8 model provided by an embodiment of the present invention.

[0056] Reference numerals:

[0057] 10-acquisition module, 20-identification module, 30-tracking module, 40-violation prompt module;

[0058] 130 - processor, 131 - memory, 132 - bus, 133 - communication interface. DETAILED DESCRIPTION

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0060] To facilitate understanding of this embodiment, the application scenarios and design concepts of the embodiment of this application are briefly introduced below.

[0061] Existing technologies mostly use a single data source and lack the ability to integrate multimodal data, resulting in a high false alarm rate and insufficient real-time performance.

[0062] Based on this, the embodiments of the present application provide a method, device, electronic device, and storage medium for identifying violations by river vessels.

[0063] Example 1

[0064] This application provides a method for identifying illegal river vessels, combining Figure 1 As shown, the method includes:

[0065] S110: Obtain an image frame to be recognized.

[0066] S120 uses the improved YOLOv8 model to perform target detection on the image frame, identify and locate the ship, and obtain the target recognition result.

[0067] S130, based on the target recognition result, continuously track the vessel using the SORT or DeepSORT algorithm, associate the detected target vessel with the targets in the tracking list using the Hungarian algorithm, and update the motion trajectory and motion parameters.

[0068] S140: If it is determined that the target vessel has violated the rules based on the motion parameters, motion trajectory and set rules of the target vessel, a prompt message of the violation is returned.

[0069] Among them, the backbone network of YOLOv8 uses the SwinTransformer module to replace the original C2f module, and the convolutional layer of the backbone network also includes multiple parallel MaxPool operations combined with the CSPN structure.

[0070] The river vessel violation identification method provided in this application uses an improved YOLOv8 model and a SwinTransformer module to enhance feature extraction accuracy, combines parallel MaxPool and CSPN structures to achieve multi-scale feature fusion, and improves the accuracy of vessel monitoring. It then combines the SORT or DeepSORT algorithm with the Hungarian algorithm to achieve stable target tracking, combines the recognition box with the actual distance mapping to estimate the speed, and accurately determines violations such as illegal parking and crossing the boundary, thereby realizing the precision, automation and intelligence of the supervision process, effectively solving the pain points of low efficiency and insufficient coverage of traditional manual supervision, and providing efficient and reliable technical support for smart river management.

[0071] In combination with the first aspect, step S110 includes:

[0072] S010 , in response to a video stream acquisition instruction, acquiring an IP address carried in the instruction.

[0073] During the process of ship identification and management, the video stream of the management area is first obtained. Since there are multiple management areas and the number of cameras used for monitoring in each management area is also multiple, when obtaining the video stream of a camera in a certain area, it is necessary to use the unique identification tag related to the camera, such as the camera's unique identification code, unique number, IP address, etc.

[0074] S020, establishing a communication connection between the control unit and the camera based on the IP address; the camera corresponds to the IP address.

[0075] In this embodiment, the IP address is used as a unique identification tag, and a communication connection is established with the camera corresponding to the IP address by using the IP address carried in the instruction issued by the administrator to facilitate communication and obtain video streams.

[0076] S030, receiving the video stream captured by the camera, and accelerating the video stream using the deepstream method.

[0077] DeepStream is an open source framework provided by NVIDIA, specifically designed to optimize AI reasoning and video stream processing. DeepStream is used to accelerate video stream processing and provide high-performance support for subsequent frame capture and analysis.

[0078] S040: performing frame interception on the accelerated video stream to obtain a plurality of continuous image frames.

[0079] This embodiment quickly establishes a communication connection with the camera based on the IP address to ensure the stability of video stream transmission, then fully utilizes GPU resources to significantly improve the efficiency of video stream processing, and then decomposes the video stream into continuous image frames to facilitate subsequent frame-by-frame analysis and processing.

[0080] Combined with the first aspect, the improved YOLOv8 model includes: backbone network, feature fusion network and detection head.

[0081] The backbone network uses the Swin Transformer module to replace the original C2f module, and adds multiple MaxPool operations in parallel in the convolutional layer, combined with the CSPN structure to achieve multi-scale feature extraction;

[0082] The feature fusion network uses the PANet structure to perform multi-level feature fusion;

[0083] The detection head adopts a decoupled head structure to separate the classification and regression tasks.

[0084] Combine Figure 4 The schematic diagram of the architecture of the YOLOv8 model in the related art shown in the figure, the module shown in the dotted box is the module to be replaced; combined with Figure 5 The schematic diagram of the improved YOLOv8 model architecture provided by this embodiment shows that the two C2f modules in the original YOLOv8 model are replaced by the Swin Transformer module, and the accuracy of feature extraction is further improved through the attention mechanism. Similarly, the pooling layer module in the backbone network is improved, and multiple parallel MaxPool operations are added to a series of convolutions, and combined with the CSPN structure to achieve effective extraction and fusion of multi-scale features. Figure 7 shown.

[0085] Combined with the first aspect, the improved YOLOv8 model also includes: an SPP layer, which is added to the backbone network to convert feature maps of any size into feature vectors of fixed size to avoid image scaling distortion.

[0086] Combine Figure 6 As shown, in this embodiment, the SPP layer is an SPP module, which can convert feature maps of any size into feature vectors of fixed size, effectively avoiding image distortion caused by operations such as image cropping and scaling, and solving the problem of repeated feature extraction of images by convolutional neural networks.

[0087] During the training process of the improved YOLOv8 model, the OpenVINO toolkit is used to perform post-training quantization on the model to optimize computing power usage.

[0088] In combination with the first aspect, step S120 includes:

[0089] S121 , for each image frame, detecting the position of the target vessel using the SORT or DeepSORT algorithm based on the target recognition result, and generating a detection frame containing the position information.

[0090] S122, using the Hungarian algorithm to match the detection frame of the current frame with the predicted position of the target in the tracking list, calculate the similarity between the detection frame and the tracked target, and obtain a matching result.

[0091] S123: Associating the detection target with the tracking target based on the matching result.

[0092] S124: For the successfully associated tracking target, use Kalman filtering to predict its motion state in the next frame, and update the position and speed information in the tracking list.

[0093] S125, for the unassociated new detection target, initialize a new tracker and add it to the tracking list.

[0094] S126 continuously outputs the motion trajectory of each target in the tracking list and marks the target ID to achieve continuous tracking across frames to update the motion trajectory.

[0095] In combination with the first aspect, before step S140, the method further includes:

[0096] S1401: Send a preset rule acquisition request to the management platform, where the request carries the target recognition result and the updated motion trajectory and motion parameters of the target vessel.

[0097] S1402: Receive preset rules returned by the management platform.

[0098] In this embodiment, the preset rules can be pre-stored on the server or configured by the administrator on the management platform based on the current situation. This linkage between the server and the management platform not only leverages the stability of pre-stored rules but also leverages the flexibility of dynamically configured rules to achieve efficient management of complex scenarios. This mechanism is particularly suitable for areas requiring rapid response and dynamic adjustment, such as river monitoring, traffic management, and security systems.

[0099] For example, the server pre-stores a set of default rules that apply to general situations. These rules can be designed based on historical data, industry standards, or common scenarios, such as "All ships must not exceed 5 kilometers per hour in restricted areas." Administrators can customize rules on the management platform based on current actual conditions (such as special events, environmental changes, etc.). New rules are transmitted to the server through the management platform and overwrite or supplement the original preset rules. In the event of an emergency (such as flooding or construction), administrators can set temporary rules through the management platform (such as "Prohibit all ships from entering Section A"), which will be immediately synchronized to the server and applied to subsequent vessel monitoring and alarm systems.

[0100] S1402: Receive preset rules returned by the management platform.

[0101] In combination with the first aspect, after step S140, the method further includes:

[0102] S141, collects statistics on violation data and classifies them by violation period, monitoring area and vessel type.

[0103] S142, performing correlation analysis on the frequency of occurrence of violation types and the monitoring area, and generating a violation distribution map, wherein the distribution map displays the spatiotemporal distribution of the violation and its correlation with the vessel type.

[0104] S143, based on the distribution map and correlation analysis results, outputs targeted management strategies, including optimizing patrol periods, adjusting key monitoring areas, or restricting navigation rights for specified vessel types.

[0105] This embodiment significantly optimizes river management through multi-dimensional analysis of violation data: based on the classification and statistics of violation data based on time period, region and vessel type, it accurately identifies high-frequency violation time periods, key monitoring areas and high-risk vessel types, and provides a scientific basis for resource allocation; the generated violation distribution map intuitively presents the temporal and spatial patterns of violations and their association with vessel types, quickly locates the source of the problem (such as a certain type of vessel crossing the boundary in a narrow river section), and supports the formulation of targeted rules such as time restrictions and regional control; by outputting strategies such as optimizing patrol schedules, adjusting monitoring priorities, and restricting the authority of designated vessels, it promotes the transition from "comprehensive supervision" to "precise control", reduces redundant manpower and equipment costs, and improves the efficiency of violation response. This solution replaces the traditional experience model with data-driven decision-making to achieve dynamic and refined river supervision.

[0106] It is understandable that, corresponding to steps S110-S140 provided in this application, this application also provides a method for identifying river vessel violations, which is applied to a management platform, and the management platform provides an interactive interface; the method includes:

[0107] Receive the preset rule acquisition request sent by the server.

[0108] The interactive interface is rendered based on the image frame in the acquisition request to generate an image display interface.

[0109] In response to a user's touch or input operation on a specified component, a preset rule is constructed and returned to the server.

[0110] It is understandable that constructing a preset rule can be to construct a violation area, and the violation area can be demarcated based on the input area boundary points, such as constructing the area endpoints as a percentage of the image width and height, so as to achieve the same effect in video streams of different resolutions. For example, the administrator edits on the first component and enters the width and height percentages, such as entering 30% width and 40% height. As another feasible method, the management interface also provides a second component, which can be used to adjust the percentage by dragging the component.

[0111] Pre-configured rules can also include vessel loitering violations. This can be monitored by defining a violation zone and the length of time a vessel remains or moves within it. If a vessel remains within the zone for longer than specified in the rules, an alarm is triggered.

[0112] Building pre-set rules can also include detecting illegal parking violations. In this case, the violation area can be mapped and monitored by monitoring the vessel's presence in the violation area. If any, an alarm will be triggered.

[0113] Similarly, it also includes: ship approaching detection, by real-time tracking the ship's position and comparing the distance with the configured route segments, obtaining the warning distance between the center of the ship and the river segment based on the detection frame and the configured tolerance. If the distance is too close, an alarm will be issued.

[0114] It also includes ship entry and exit detection: entry and exit determination lines and entry and exit direction lines are drawn. When a ship approaches the determination line, the current position of the ship is compared with the earliest identified position. After obtaining the ship's direction vector, it is compared with the entry and exit direction line to obtain the ship's entry and exit information.

[0115] As an example, simple rules are automatically configured by performing edge analysis and other feature recognition on a video stream of a river. For cases where a ship is docking, edge analysis of the river scene in the video stream is performed. After line segment fitting and noise removal, the river edge is obtained and automatically configured as an edge where the ship should not dock. Furthermore, based on collected port image data, the port locations in the video are analyzed and deemed as suitable for docking, and then truncated. For ship entry and exit detection, the camera must first be enabled for entry and exit detection. Then, by counting the locations where the recognition frame appears and disappears, as well as the aspect ratio of the recognition frame, entry and exit determination lines and entry and exit direction lines are set for the river area. For the automatic setting of no-parking and wandering zones, similarly to ship docking detection, the endpoints of each edge segment are connected based on their proximity to each other to determine the area shape. Similarly, visual recognition is used to avoid scenes such as ports where docking is required.

[0116] As will be appreciated, once the violation rules are constructed on the interactive interface, they can be stored in a designated storage space and can be added, modified, or deleted. Furthermore, the interactive interface also includes a display component that processes and displays the received information, such as highlighting the offending vessel on the monitoring interface or triggering an alarm system.

[0117] In the second aspect, the embodiment of the present application provides a device for identifying violations of river vessels. In combination with the second aspect, such as Figure 2 As shown, the device includes: an acquisition module 10, an identification module 20, a tracking module 30 and a violation prompt module 40.

[0118] The acquisition module 10 is used to acquire an image frame to be recognized.

[0119] The recognition module 20 is used to use the improved YOLOv8 model to perform target detection on the image frame, identify and locate the ship, and obtain a target recognition result.

[0120] The tracking module 30 is used to continuously track the ship using the SORT or DeepSORT algorithm based on the target recognition result, associate the detected target ship with the targets in the tracking list through the Hungarian algorithm, and update the motion trajectory and motion parameters.

[0121] The violation prompt module 40 is used to return a violation prompt message if it is determined that the target vessel has violated the rules based on the motion parameters, motion trajectory and set rules of the target vessel.

[0122] Among them, the backbone network of the improved YOLOv8 model adopts the following Figure 7 The SwinTransformer module shown replaces the original C2f module, and the convolutional layer of the backbone network also includes multiple parallel MaxPool operations combined with the CSPN structure. The CSPN structure reduces the amount of computation by segmenting and reusing feature maps while maintaining the expressive power of the model. Combined with MaxPool, it can effectively extract and fuse multi-scale features. Figure 7 In, the zth l-1 The input of the layer is processed by the LN layer and W-MSA to obtain the first component, and z l-1 The input of the layer is taken as the second component, and the first component and the second component are summed to obtain the zth component. l-1 The output intermediate value of the layer is then input into the LN layer and the MLP layer to obtain the third component, and the output intermediate value is used as the fourth component. The fourth component and the third component are summed to obtain the zth component. l-1 The final output of the layer is passed to the zth l Layer, repeat the above steps to process the data and get the zth layer. l The output of the layer is passed to the zth l-1 Layer. That is, except for the first layer, the input of each layer is the final output value obtained by summing the intermediate output value of the previous layer and the intermediate output value after processing by the LN layer and the MLP layer. This is just an example and will not be explained in detail.

[0123] In a third aspect, the present application provides an electronic device, Figure 3 As shown, the electronic device includes a memory 131 and a processor 130. The memory 131 is used to store computer programs, and the processor 130 runs the computer programs to enable the electronic device to perform the above method.

[0124] Further, combined with Figure 3 The electronic device shown further includes a bus 132 and a communication interface 133 , and the processor 130 , the communication interface 133 and the memory 131 are connected via the bus 132 .

[0125] The memory 131 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is achieved through at least one communication interface 133 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 132 may be an ISA bus, a PCI bus, or an EISA bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0126] The processor 130 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 130 or by software instructions. The processor 130 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 131, and processor 130 reads information in memory 131 and, in conjunction with its hardware, completes the steps of the method of the aforementioned embodiment.

[0127] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the above-mentioned method is executed.

[0128] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0129] In addition, in the description of the embodiments of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to mechanical connections or electrical connections; they may refer to direct connections or indirect connections through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0130] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0131] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0132] Finally, it should be noted that the above embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for identifying illegal activities of river vessels, characterized in that: Applied to the server; the method includes: Obtaining the image frame to be recognized; Using an improved YOLOv8 model to perform target detection on the image frame, identify and locate the ship, and obtain a target recognition result; Based on the target recognition results, the SORT or DeepSORT algorithm is used to continuously track the vessel, the Hungarian algorithm is used to associate the detected target vessel with the targets in the tracking list, and the motion trajectory and motion parameters are updated; If it is determined that the target vessel has violated the rules based on the target vessel's motion parameters, motion trajectory, and set rules, a prompt message indicating the violation is returned; The backbone network of the improved YOLOv8 model uses the SwinTransformer module to replace the original C2f module, and the convolutional layer of the backbone network also includes multiple parallel MaxPool operations combined with the CSPN structure.

2. The method according to claim 1, characterized in that The steps of obtaining the image frame to be recognized include: In response to a video stream acquisition instruction, acquiring an IP address carried in the instruction; Establishing a communication connection between the server and the camera based on the IP address; the camera corresponds to the IP address; Receive the video stream captured by the camera and accelerate the video stream using the deepstream method; Frame capture is performed on the accelerated video stream to obtain a plurality of continuous image frames.

3. The method according to claim 1, characterized in that The improved YOLOv8 model includes: The backbone network uses the Swin Transformer module to replace the C2f module, and adds multiple MaxPool operations in parallel in the convolutional layer, combining the CSPN structure to achieve multi-scale feature extraction; Feature fusion network, using PANet structure for multi-level feature fusion; The detection head adopts a decoupled head structure to separate the classification and regression tasks.

4. The method according to claim 1, wherein The improved YOLOv8 model also includes: The SPP layer is added to the backbone network to convert feature maps of any size into feature vectors of fixed size to avoid image scaling distortion.

5. The method according to claim 1, wherein Based on the target recognition result, the steps of continuously tracking the vessel using the SORT or DeepSORT algorithm, associating the detected target vessel with the targets in the tracking list using the Hungarian algorithm, and updating the motion trajectory and motion parameters include: For each image frame, based on the target recognition result, the target vessel position is detected using the SORT or DeepSORT algorithm to generate a detection frame containing the position information; The Hungarian algorithm is used to match the detection frame of the current frame with the predicted position of the target in the tracking list, and the similarity between the detection frame and the tracked target is calculated to obtain the matching result; Associating the detection target with the tracking target based on the matching result; For the successfully associated tracking target, use Kalman filtering to predict its motion state in the next frame and update the position and velocity information in the tracking list; For unassociated new detection targets, initialize a new tracker and add it to the tracking list; Continuously output the motion trajectory of each target in the tracking list and mark the target ID to achieve continuous tracking across frames to update the motion trajectory.

6. The method according to claim 1, characterized in that The server is in communication with the management platform; If it is determined based on the motion parameters, motion trajectory, and set rules of the target vessel that the vessel has violated the rules, before returning the prompt information of the violation, the method further includes: Sending a preset rule acquisition request to the management platform, wherein the request carries the image frame, the target recognition result, the updated motion trajectory, and the motion parameters; Receive the preset rules returned by the management platform.

7. The method according to claim 1, characterized in that If it is determined based on the motion parameters, motion trajectory and set rules of the target vessel that the vessel has violated the rules, after returning the prompt information of the violation, the method further includes: Collect and compile violation data, categorizing them by violation period, monitoring area, and vessel type; Conduct correlation analysis between the frequency of violations and the monitored areas to generate a violation distribution map showing the spatial and temporal distribution of violations and their correlation with vessel types; Based on the distribution map and correlation analysis results, targeted management strategies are output, including optimizing patrol periods, adjusting key monitoring areas, or restricting navigation rights of specified vessel types.

8. A device for identifying illegal activities of vessels in a river, characterized in that: The device comprises: An acquisition module, used for acquiring an image frame to be identified; a recognition module, configured to perform target detection on the image frame using an improved YOLOv8 model, identify and locate the vessel, and obtain a target recognition result; A tracking module is configured to continuously track the vessel using a SORT or DeepSORT algorithm based on the target recognition result, associate the detected target vessel with the targets in the tracking list using a Hungarian algorithm, and update the motion trajectory and motion parameters; A violation prompt module is used to return a prompt message of the violation if it is determined that the vessel has violated the rules based on the motion parameters, motion trajectory and set rules of the target vessel; The backbone network of the improved YOLOv8 model uses the SwinTransformer module to replace the original C2f module, and the convolutional layer of the backbone network also includes multiple parallel MaxPool operations combined with the CSPN structure.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images