Ship trajectory filing method and device, electronic equipment and readable storage medium
Through the ship number ORC identification system and the ship ReID analysis system combined with the river channel scene ship detection model, the automatic archiving of ship trajectory is realized, solving the problem of inefficient use of AIS systems by inland shipping regulators, and improving supervision efficiency and data reliability.
Patent Information
- Application Number
- CN202411980915.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-05-16
AI Technical Summary
Inland shipping regulatory agencies find it difficult to use maritime AIS systems conveniently, resulting in inefficiency and frequent inspection errors. The reliability and accuracy of AIS data are limited. Ships often cannot obtain AIS signals in inland waterways, and the lack of effective ship video and photo information is not conducive to regulatory evidence collection and law enforcement.
The ship close-up image information is obtained through the ship number ORC identification system, the ship video information is obtained and pre-processed through the ship ReID analysis system, and the river scene ship detection model is called to generate ship position prediction data to realize automatic archiving of ship trajectory.
It improves the efficiency of ship supervision, reduces the workload of manual processing, and provides accurate recording of ship driving trajectories and basic data for regulatory evidence collection.
Smart Images
Figure CN120011465A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of ship supervision technology, and in particular to a ship track archiving method, device, electronic equipment and readable storage medium. Background Art
[0002] Inland waterway regulatory agencies often cannot conveniently use the maritime AIS system and have to use manual inspections and manual records, resulting in low efficiency and frequent inspection errors. The reliability and accuracy of AIS data are also limited. Ships often cannot obtain AIS signals in inland waterways, and radar monitoring is limited by the layout and range of equipment and cannot cover the entire waterway. AIS data also lacks effective ship video and photo information, which is not conducive to regulatory evidence collection during ship management and effective ship supervision and law enforcement.
[0003] Therefore, a ship track archiving method, device, electronic device and readable storage medium are provided to solve the above problems. Summary of the invention
[0004] The main purpose of the present invention is to solve the problem in the prior art that inland waterway regulatory agencies are often unable to conveniently use the maritime AIS system and have to use manual inspections and manual records, resulting in low efficiency and frequent inspection errors; the reliability and accuracy of AIS data are also limited, and ships often cannot obtain AIS signals in inland waterways. Radar monitoring is limited by the layout and range of equipment and cannot cover the entire waterway. AIS data also lacks effective ship video and photo information, which is not conducive to regulatory evidence collection during ship management and effective ship supervision and law enforcement.
[0005] A first aspect of the present invention provides a ship track archiving method, the ship track archiving method comprising: Obtain close-up image information of the ship through the ship number ORC identification system; Obtaining ship video information, preprocessing the ship video information through a ship ReID analysis system to generate first image information, and calling a river scene ship detection model to generate ship position prediction data according to the first image information; Get the ship track archiving command, and traverse all the point ship data information in the ship number ORC identification system and the ship ReID analysis system; Determine whether the selected point has a parent point; if so, query the ship records within the specified time period of the parent point; Obtain the most similar ship record information based on the ship number and ship ReID information; The most similar ship record information is added to the ship track, the current point ship record is used as the selected point, and the ship track information is saved.
[0006] Optionally, the acquiring of close-up image information of a ship through a ship ORC identification system includes: Acquire panoramic image information and close-up image information of the ship, wherein the close-up image information includes bow image information, midship image information, and stern image information; The ship ORC identification system is called to analyze the panoramic image information, bow image information, midship image information and stern image information through the ship ORC identification system, extract the ship number information, and record the image capture time information and image capture point information.
[0007] Optionally, the acquiring of ship video information, preprocessing the ship video information by a ship ReID analysis system to generate first image information, and calling a river scene ship detection model to generate ship position prediction data according to the first image information includes: Obtain ship video information, convert the video information into data format, and generate video frame data; Acquire video frame data, perform image enhancement operation on the video frame data, and generate first image information; The river scene vessel detection model is called to generate vessel position prediction data according to the first image information.
[0008] Optionally, the training process of the river scene vessel detection model includes: Obtain real samples of the same ship captured by different cameras as basic training samples; Build an SSD structure model, put the basic training samples into the SSD structure model, output the coordinate information of the ship in the river scene and the ReID value information of the ship, and the ReID value information is 512-dimensional floating point feature value information; The weighted sum of position error and confidence error is used for loss function processing; Performing image enhancement operation on the obtained river scene image; Train ship detection in river scenes. After a stable model is obtained, add a ship feature value extraction branch to ship detection and continue training until the network converges.
[0009] Optionally, the SSD structure model includes: a ResNet18 structure and a CenterNet structure; The SSD structure model building process includes: Modify the input resolution of the ResNet18 structure; Modify the number of network channels of the ResNet18 structure and CenterNet structure; The network modules of the ResNet18 structure and the CenterNet structure are fused.
[0010] Optionally, calling a river scene vessel detection model to generate vessel position prediction data according to the first image information includes: Generate multiple prediction frames according to the first image information, determine the confidence value of each prediction frame according to the category confidence, and filter the prediction frame belonging to the background; Filter prediction boxes whose confidence is lower than the preset value according to the confidence threshold; Decode the prediction frame and perform clip processing; Get the real position parameters according to the inspection box; Sort in descending order according to confidence, and only keep the top-k prediction boxes; Perform the NMS algorithm to filter the prediction boxes whose overlap is greater than the preset value to obtain the detection results; Through forward reasoning, all ship position prediction data and ship feature values on the first image information are obtained.
[0011] Optionally, if the selected point does not have a parent point, the current ship track search and archiving will be terminated, and a new ship track search and archiving will be started.
[0012] A second aspect of the present invention provides a ship track archiving device, comprising: A close-up image information acquisition module is used to acquire close-up image information of a ship through a ship number ORC recognition system; A ship position prediction module is used to obtain ship video information, pre-process the ship video information through a ship ReID analysis system to generate first image information, and call a river scene ship detection model to generate ship position prediction data according to the first image information; The ship track archiving module is used to obtain the ship track archiving command and traverse all the point ship data information in the ship number ORC identification system and the ship ReID analysis system; The first judgment module is used to judge whether the selected point has a parent point; if so, query the ship records within the specified time period of the parent point; The ship record information acquisition module is used to obtain the most similar ship record information according to the ship number and ship ReID information; The ship track information saving module is used to add the most similar ship record information to the ship track, take the current point ship record as the selected point, and save the ship track information.
[0013] The close-up image information acquisition module includes: an image acquisition unit and a spatiotemporal information acquisition unit; An image acquisition unit, used to acquire panoramic image information and close-up image information of the ship, wherein the close-up image information includes bow image information, midship image information and stern image information; The spatiotemporal information acquisition unit is used to call the ship ORC identification system, analyze the panoramic image information, bow image information, midship image information and stern image information through the ship ORC identification system, extract the ship number information, and record the image capture time information and image capture point information; The ship track archiving module includes: a video frame data acquisition unit, a first image information acquisition unit and a ship position prediction data unit; A video frame data acquisition unit is used to acquire ship video information, convert the video information into a data format, and generate video frame data; A first image information acquisition unit, used to acquire video frame data, perform image enhancement operation on the video frame data, and generate first image information; The ship position prediction data unit is used to call the river scene ship detection model to generate ship position prediction data according to the first image information.
[0014] A third aspect of the present invention provides an electronic device, the electronic device comprising a memory and at least one processor, the memory storing instructions; The at least one processor calls the instructions in the memory to enable the electronic device to execute the various steps of the ship track archiving method as described above.
[0015] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, and the instructions, when executed by a processor, implement the various steps of the ship track archiving method as described above.
[0016] In the technical solution of the present invention, the extraction of spatiotemporal information of ship travel, including ship number, ship ReId and specific travel track of the ship, is realized through the fusion analysis of checkpoints, videos of ships along the river and radar data; the ship travel track is searched and archived through the data extracted by the ship number OCR recognition system and the ship ReId intelligent analysis system, providing basic data support for ship management and backtracking; the automatic archiving of the ship travel track is realized through intelligent analysis and data fusion processing, which greatly improves the efficiency of ship supervision and reduces the workload of manual processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 A flow chart of a ship track archiving method provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a ship track archiving device provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention; Figure 4A flow chart for generating vessel position prediction data of the present invention; Figure 5 Flowchart for training a ship detection model for river scenes. DETAILED DESCRIPTION
[0018] The embodiment of the present invention provides a ship track archiving method, including obtaining close-up image information of a ship through a ship number ORC recognition system; obtaining ship video information, preprocessing the ship video information through a ship ReID analysis system to generate first image information, calling a river scene ship detection model to generate ship position prediction data according to the first image information; obtaining a ship track archiving command, traversing all point ship data information in the ship number ORC recognition system and the ship ReID analysis system; judging whether a selected point has an upper point; if so, querying the ship records within a specified time period of the upper and lower points; obtaining the most similar ship record information according to the ship number and the ship ReID information; adding the most similar ship record information to the ship track, taking the current point ship record as the selected point, and saving the ship track information. The present invention solves the problem in the prior art that inland shipping regulatory agencies are often unable to conveniently use the maritime AIS system, and have to use manual inspections and manual records, resulting in low efficiency and frequent inspection errors; and the reliability and accuracy of AIS data are also limited. Ships often cannot obtain AIS signals in inland waterways, and radar monitoring is limited by the layout and range of equipment and cannot cover the entire waterway. AIS data also lacks effective ship video and photo information, which is not conducive to regulatory evidence collection during ship management and is not conducive to effective ship supervision and law enforcement.
[0019] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0020] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , 4 and 5. A first embodiment of the ship track archiving method in the embodiment of the present invention comprises: The close-up image information of the ship is obtained through the ship number ORC identification system; specifically including: Acquire panoramic image information and close-up image information of the ship, wherein the close-up image information includes bow image information, midship image information, and stern image information; Calling the ship ORC identification system, analyzing the panoramic image information, bow image information, midship image information and stern image information through the ship ORC identification system, extracting the ship number information, and recording the image capture time information and image capture point information; Obtaining ship video information, preprocessing the ship video information through a ship ReID analysis system to generate first image information, and calling a river scene ship detection model to generate ship position prediction data according to the first image information; Specifically, they include: Obtain ship video information, convert the video information into data format, and generate video frame data; Acquire video frame data, perform image enhancement operation on the video frame data, and generate first image information; Calling a river scene ship detection model to generate ship position prediction data according to the first image information; including: generating multiple prediction frames according to the first image information, determining the confidence value of each prediction frame according to the category confidence, and filtering the prediction frames belonging to the background; filtering the prediction frames with confidence values lower than a preset value according to the confidence threshold; decoding the prediction frames and performing clip processing; obtaining the true position parameters according to the inspection frames; arranging in descending order according to the confidence values, and retaining only the top-k prediction frames; performing the NMS algorithm, filtering the prediction frames with overlap values greater than a preset value, and obtaining the detection results; obtaining all the ship position prediction data and ship feature values on the first image information through forward reasoning; The training process of the river scene ship detection model includes: Obtain real samples of the same ship captured by different cameras as basic training samples; Build an SSD structure model, put the basic training samples into the SSD structure model, output the coordinate information of the ship in the river scene and the ReID value information of the ship, and the ReID value information is 512-dimensional floating point feature value information; the SSD structure model includes: ResNet18 structure and CenterNet structure; the SSD structure model building process includes: modifying the input resolution of the ResNet18 structure; modifying the number of network channels of the ResNet18 structure and the CenterNet structure; fusing the network modules of the ResNet18 structure and the CenterNet structure, and using the weighted sum of the position error and the confidence error for loss function processing; Performing image enhancement operation on the obtained river scene image; Train ship detection in river scenarios. After a stable model is obtained, add a branch for extracting ship features in ship detection and continue training until the network converges. Get the ship track archiving command, and traverse all the point ship data information in the ship number ORC identification system and the ship ReID analysis system; Determine whether the selected point has a parent point; if so, query the ship records within the specified time period of the parent point; if the selected point does not have a parent point, end the search and filing of this ship track, and start a new ship track search and filing; Obtain the most similar ship record information based on the ship number and ship ReID information; The most similar ship record information is added to the ship track, the current point ship record is used as the selected point, and the ship track information is saved.
[0021] The above describes the ship track archiving method in the embodiment of the present invention. The following describes the ship track archiving device in the embodiment of the present invention. Figure 2 In the embodiment of the present invention, the ship track archiving device includes: A close-up image information acquisition module 201 is used to acquire close-up image information of a ship through a ship number ORC recognition system; The ship position prediction module 202 is used to obtain ship video information, pre-process the ship video information through the ship ReID analysis system to generate first image information, and call the river scene ship detection model to generate ship position prediction data according to the first image information; The ship track archiving module 203 is used to obtain the ship track archiving command and traverse all the point ship data information in the ship number ORC identification system and the ship ReID analysis system; The first judgment module 204 is used to judge whether the selected point has a parent point; if so, query the ship records within the specified time period of the parent point; The ship record information acquisition module 205 is used to acquire the most similar ship record information according to the ship number and the ship ReID information; The ship track information saving module 206 is used to add the most similar ship record information to the ship track, take the current point ship record as the selected point, and save the ship track information.
[0022] The close-up image information acquisition module includes: an image acquisition unit and a spatiotemporal information acquisition unit; An image acquisition unit, used to acquire panoramic image information and close-up image information of the ship, wherein the close-up image information includes bow image information, midship image information and stern image information; The spatiotemporal information acquisition unit is used to call the ship ORC identification system, analyze the panoramic image information, bow image information, midship image information and stern image information through the ship ORC identification system, extract the ship number information, and record the image capture time information and image capture point information; The ship track archiving module includes: a video frame data acquisition unit, a first image information acquisition unit and a ship position prediction data unit; A video frame data acquisition unit is used to acquire ship video information, convert the video information into a data format, and generate video frame data; A first image information acquisition unit, used to acquire video frame data, perform image enhancement operation on the video frame data, and generate first image information; The ship position prediction data unit is used to call the river scene ship detection model to generate ship position prediction data according to the first image information.
[0023] above Figure 2 The ship trajectory archiving device in the embodiment of the present invention is described in detail from the perspective of modular functional entities, and the electronic device in the embodiment of the present invention is described in detail from the perspective of hardware processing.
[0024] Figure 3 7 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 733 or data 732. Among them, the memory 720 and the storage medium 730 can be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the electronic device 700. Furthermore, the processor 710 may be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the electronic device 700.
[0025] The electronic device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 750, and / or one or more operating systems 731, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 3The structure of the electronic device shown does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.
[0026] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the ship trajectory archiving method.
[0027] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.
[0028] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.
[0029] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A ship track archiving method, characterized in that: The ship track archiving method comprises: Obtain close-up image information of the ship through the ship number ORC identification system; Obtaining ship video information, preprocessing the ship video information through a ship ReID analysis system to generate first image information, and calling a river scene ship detection model to generate ship position prediction data according to the first image information; Get the ship track archiving command, and traverse all the point ship data information in the ship number ORC identification system and the ship ReID analysis system; Determine whether the selected point has a parent point; if so, query the ship records within the specified time period of the parent point; Obtain the most similar ship record information based on the ship number and ship ReID information; The most similar ship record information is added to the ship track, the current point ship record is used as the selected point, and the ship track information is saved.
2. The ship track archiving method according to claim 1, characterized in that: The close-up image information of the ship obtained by the ship ORC identification system includes: Acquire panoramic image information and close-up image information of the ship, wherein the close-up image information includes bow image information, midship image information, and stern image information; The ship ORC identification system is called to analyze the panoramic image information, bow image information, midship image information and stern image information through the ship ORC identification system, extract the ship number information, and record the image capture time information and image capture point information.
3. The ship track archiving method according to claim 2, characterized in that: The obtaining of ship video information, preprocessing of the ship video information by a ship ReID analysis system to generate first image information, and calling a river scene ship detection model to generate ship position prediction data according to the first image information include: Obtain ship video information, convert the video information into data format, and generate video frame data; Acquire video frame data, perform image enhancement operation on the video frame data, and generate first image information; The river scene vessel detection model is called to generate vessel position prediction data according to the first image information.
4. The ship track archiving method according to claim 3, characterized in that: The training process of the river scene ship detection model includes: Obtain real samples of the same ship captured by different cameras as basic training samples; Build an SSD structure model, put the basic training samples into the SSD structure model, output the coordinate information of the ship in the river scene and the ReID value information of the ship, and the ReID value information is 512-dimensional floating point feature value information; The weighted sum of position error and confidence error is used for loss function processing; Performing image enhancement operation on the obtained river scene image; Train ship detection in river scenes. After a stable model is obtained, add a ship feature value extraction branch to ship detection and continue training until the network converges.
5. The ship track archiving method according to claim 4, characterized in that: The SSD structure model includes: ResNet18 structure and CenterNet structure; The SSD structure model building process includes: Modify the input resolution of the ResNet18 structure; Modify the number of network channels of the ResNet18 structure and CenterNet structure; The network modules of the ResNet18 structure and the CenterNet structure are fused.
6. The ship track archiving method according to claim 4, characterized in that: The calling of the river scene ship detection model to generate ship position prediction data according to the first image information includes: Generate multiple prediction frames according to the first image information, determine the confidence value of each prediction frame according to the category confidence, and filter the prediction frame belonging to the background; Filter prediction boxes whose confidence is lower than the preset value according to the confidence threshold; Decode the prediction frame and perform clip processing; Get the real position parameters according to the inspection box; Sort in descending order according to confidence, and only keep the top-k prediction boxes; Perform the NMS algorithm to filter the prediction boxes whose overlap is greater than the preset value to obtain the detection results; Through forward reasoning, all ship position prediction data and ship feature values on the first image information are obtained.
7. The ship track archiving method according to claim 1, characterized in that: The method further comprises: If the selected point does not have a parent point, the current ship track search and archiving will end, and a new ship track search and archiving will be started.
8. A ship track archiving device, characterized in that: include: A close-up image information acquisition module is used to acquire close-up image information of a ship through a ship number ORC recognition system; A ship position prediction module is used to obtain ship video information, pre-process the ship video information through a ship ReID analysis system to generate first image information, and call a river scene ship detection model to generate ship position prediction data according to the first image information; The ship track archiving module is used to obtain the ship track archiving command and traverse all the point ship data information in the ship number ORC identification system and the ship ReID analysis system; The first judgment module is used to judge whether the selected point has a parent point; If yes, query the ship records within the specified time period of the above point; The ship record information acquisition module is used to obtain the most similar ship record information according to the ship number and ship ReID information; The ship track information saving module is used to add the most similar ship record information to the ship track, take the current point ship record as the selected point, and save the ship track information.
9. An electronic device, comprising a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the electronic device to execute each step of the ship track archiving method as described in any one of claims 1-7.
10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the ship track archiving method as described in any one of claims 1 to 7 are implemented.