Seaman allocation condition automatic analysis method based on video snapshot and fishing boat positioning

Abstract

The invention discloses a sailor allocation condition automatic analysis method based on video snapshot and fishing boat positioning, and the method comprises the following steps: S1, before a fishing boat leaves a port, daily constructing a fishery sailor information base, filling a port leaving report before a captain leaves the port, and recognizing the departure of the fishing boat; s2, when the fishing boat is on the sea, automatically capturing sailor information through video equipment and uploading the sailor information to a server side, analyzing sailors by the server side, and generating an on-ship sailor information list; s3, carrying out business analysis on the fishing boat in the sea or before the fishing boat enters the port, and generating early warning information in real time; and S4, when the fishing boats enter the port, after the fishing boats are recognized to enter the port, reminding is carried out according to the fishing boat research and judgment condition, boarding inspection is arranged if abnormal early warning exists, and the actual condition is checked. According to the invention, the port entering and leaving report of the fishing boat and the crew allocation condition can be analyzed, the filling workload of a captain can be reduced, and the seaworthiness condition of the fishing boat can be intelligently warned.

Description

Technical Field

[0001] The present invention relates to the technical field of automated analysis, and in particular to an automated analysis method for crew manning conditions based on video capture and fishing vessel positioning. Background Art

[0002] In order to strengthen the management of fishing vessels entering and leaving fishing ports and implement the main responsibility for production safety, the Ministry of Agriculture and Rural Affairs implemented the reporting system for fishing vessels entering and leaving fishing ports from August 1, 2019, requiring large and medium-sized (12 meters or more in length) marine fishing vessels entering and leaving China's fishing ports to report to the management department of the intended fishing port before entering and leaving the fishing port. The reporting matters include manning. The system also requires fishing vessels to always keep the onboard communication and navigation terminal equipment in normal working condition, and shall not deliberately shield, shut down, or damage it to ensure that fishing vessels can accurately locate. The "Regulations on the Management of Fishery Crews of the People's Republic of China" divides fishery crews into duty crews and ordinary crews, and requires marine fishing vessels to meet the minimum manning standards for duty crews of marine fishing vessels stipulated in the regulations.

[0003] Although the state has gradually strengthened the safety requirements for fishing vessels and explicitly required the implementation of a reporting system for fishing vessels entering and leaving fishing ports, in the actual supervision and inspection process, it is still found from time to time that the reported crew members are inconsistent with the actual crew members on board, the crew certificates do not meet the requirements of the job crew members, and even overcrowding. At the same time, with the continuous development of information technology, the continuous improvement of management policies, and the continuous upgrading of fishing vessel equipment, many fishing vessels have the relevant conditions for automated collection, but there are still certain inconsistencies and errors in the captain's reporting. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an automated analysis method for crew staffing based on video capture and fishing vessel positioning, which can analyze fishing vessel entry and exit reports and crew staffing situations, reduce the captain's workload for filling out reports and provide intelligent early warning of fishing vessel seaworthiness.

[0005] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows.

[0006] An automated analysis method for crew manning based on video capture and fishing vessel positioning comprises the following steps:

[0007] S1. Before the fishing vessel leaves the port: including daily construction of the fishery crew information database, the captain filling out the port departure report before leaving the port, and identifying the fishing vessel leaving the port;

[0008] S2. When the fishing boat is at sea: When the fishing boat leaves the fishing port, the crew information is automatically captured by the video equipment and uploaded to the server. The server analyzes the crew and generates a list of crew information on board;

[0009] S3. Before fishing boats are at sea or in port: conduct business analysis and generate early warning information in real time;

[0010] S4. When a fishing boat enters the port: After identifying the fishing boat entering the port, a reminder will be given based on the situation of the fishing boat. If there is an abnormal warning, an onboard inspection will be arranged to check the actual situation.

[0011] Preferably, the step S1 specifically includes the following steps:

[0012] S11. During the daily operation of the business system, the crew's photos, basic information, and certificate information are collected through the fishery crew certification data to build a fishery crew information database to achieve "one file for each crew member";

[0013] S12. Before the fishing vessel leaves the port, the captain can bind the crew members on board in advance, or bind them when filling in the report; select the fishing port, departure time, and crew members on board; the captain can directly use the crew information filled in or identified last time, or can use the crew information automatically captured and generated by the system as the crew list to be filled in;

[0014] S13. The system identifies the fishing boat leaving the port through the fishing boat's position and electronic fence, and by judging the latest and last reported positions, and marks the fishing boat as leaving the port.

[0015] Preferably, the step S2 specifically includes the following steps:

[0016] S21. The system marks the fishing boat as being at sea by its position. When the fishing boat is operating at sea, the onboard cameras at different parts of the fishing boat periodically capture the facial photos of the crew members on board and upload them to the server via the marine broadband. When there is no network signal, the photos are cached and uploaded after the network is restored.

[0017] S22. The server receives the uploaded captured face photo and obtains the nearest fishing boat position information according to the capture time, which is used for subsequent correlation analysis to determine whether the crew is actually operating at sea following the fishing boat when the photo was captured;

[0018] S23. The server calls the image comparison and search interface to retrieve the crew's identity information and certificate information, and uses the captured photo's face as the primary key to create a list of crew information on board; when multiple photos of a crew are captured, the photo with the furthest position from the departure fishing port after the associated ship position is used first;

[0019] S24. The server associates the identified crew information with certificates and marks the official crew members and ordinary crew members. If there are faces that are not associated with crew certificates, they will be marked and an early warning will be generated that "the ship is suspected to have uncertified crew members operating at sea."

[0020] Preferably, in step S3, since there may be no network when the fishing boat is at sea, the system generates a real-time warning when the fishing boat has a network; if there is no network, when the mobile communication network is restored before entering the port, historical snapshots are uploaded and business analysis is carried out.

[0021] Preferably, the step S3 specifically includes the following steps:

[0022] S31.According to the captain of the fishing vessel and the total power of the main engine, obtain the standard for the manning of marine fishing vessels;

[0023] S32. Analyze whether the fishing vessel crew is fully staffed according to the marine fishery position crew staffing standards. When the crew certificate contains multiple positions such as driving and engine, only one position can be used for standard matching, but all positions can be cycled in sequence for matching calculations. All crew members are compared with the standards according to their positions and marked after meeting the requirements; if all personnel and all positions are traversed in sequence and still do not meet the standards, they will be marked and an early warning of "the ship is suspected of lacking marine position crew staffing" will be generated;

[0024] S33. Compare the automatically generated list of crew information on board with the personnel reported by the captain when leaving the port, analyze whether the captured personnel are consistent with the reported personnel, and if there is any inconsistency, mark it and generate an early warning that "the reported crew of the ship is suspected to be inconsistent with the actual crew";

[0025] S34. After all non-repetitive facial information captured during the voyage is recognized, the total number of people is used to determine the number of crew members on the fishing vessel. The number of people is compared with the approved number of people inspected by the ship to analyze whether there is overcrowding. If there is overcrowding, it will be marked and a warning will be generated that "the number of crew members on board the ship is suspected to exceed the approved number."

[0026] Preferably, the step S4 specifically includes the following steps:

[0027] S41. When a fishing boat enters the port, it is identified through electronic fences or port videos, and a reminder of the fishing boat entering the port is generated, and the end of the voyage is marked;

[0028] S42. Check whether there are any abnormal marks on the fishing vessels entering the port on this voyage and whether relevant warning information is generated;

[0029] S43. If the above situation information exists, the management personnel will arrange boarding inspection, and confirm, track and close the warning information based on the inspection results; if the above situation information does not exist, it means that there is no abnormal crew configuration on this voyage.

[0030] Preferably, the specific method for identifying the departure and arrival of fishing boats is as follows: the fishing boat trajectory is defined as a spatiotemporal sequence of the position of a fishing boat within a period of time, which is described as: T = (P0, P1, ..., Pn ), where n is the number of fishing boat track points collected during this period; P represents the track point of the fishing boat track, and the track point includes but is not limited to longitude, latitude and time; the system determines P in turn through the spatial algorithm n Is it within the electronic fence of all fishing ports? If it is identified in any fishing port, the calculation ends and P is associated n-1 ; such as P n In Hong Kong, P n-1 If it is not in port, it is identified as a fishing boat entering the port; n Not in Hong Kong, P n-1 If in port, it is identified as a fishing vessel leaving the port.

[0031] Due to the adoption of the above technical scheme, the technical progress achieved by the present invention is as follows.

[0032] The present invention automatically captures the facial information of the crew members on board through the onboard video equipment during the voyage of the fishing boat, and automatically associates the fishing boat's position information, thereby realizing the analysis and judgment of the actual sea operation situation of the fishing boat crew members.

[0033] The present invention can automatically generate a list of onboard crew information by performing image retrieval on recognized facial information and associating crew information and certificate information, and verify the certification status of the fishing boat crew and whether the person and the certificate are consistent.

[0034] The present invention can analyze the staffing situation of the job crew according to the staffing standard of the job crew of the marine fishing vessel, and give early warning for the situation of incomplete staffing, especially for the multi-job crew, and verify through circular matching.

[0035] The present invention can automatically summarize the number of crew members identified in the current voyage, and analyze the overloading situation of the crew members of the fishing vessel by comparing and checking the approved number of people.

[0036] The present invention can remind fishing boats with abnormal marks and early warnings in real time by identifying fishing boats entering the port.

[0037] The present invention realizes the automated analysis of the real situation of the crew of a fishing vessel when operating at sea, thereby preventing false reports, erroneous reports, inconsistency between reported crew and actual crew, etc.; and can reduce the reporting work of the captain when leaving the port through this automated method, thereby enhancing the management department's ability to analyze the seaworthiness of the crew of the fishing vessel and the efficiency of handling the situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] An automated analysis method for crew manning based on video capture and fishing vessel positioning, combined with Figure 1 As shown, the following steps are included:

[0041] S1. Before the fishing boat leaves the port: including the daily construction of the fishery crew information database, the captain filling out the port departure report before leaving the port, and identifying the fishing boat leaving the port. The specific steps include:

[0042] S11. During the daily operation of the business system, the crew's photos, basic information, and certificate information are collected through the fishery crew certification data to build a fishery crew information database and realize "one file for each crew member".

[0043] S12. Before the fishing vessel leaves the port, the captain can bind the crew members on board in advance, or bind them when filling in the report; select the fishing port to leave, the departure time, and the crew members on board; the captain can directly use the crew information filled in or identified last time, or use the crew information automatically captured and generated by the system as the crew list to fill in.

[0044] S13. The system identifies the fishing boat leaving the port through the fishing boat's position and electronic fence, and by judging the latest and last reported positions, and marks the fishing boat as leaving the port.

[0045] S2. When the fishing boat is at sea: When the fishing boat leaves the fishing port, the crew information is automatically captured by the video equipment and uploaded to the server. The server analyzes the crew and generates a list of crew information on board. The specific steps include:

[0046] S21. The system marks the fishing vessel as being at sea through the vessel's position. When the fishing vessel is operating at sea, the onboard cameras in the cockpit, deck, cabin and other parts will capture facial photos of the crew members on board from time to time, and upload them to the server through marine broadband; when there is no network signal, the photos are cached and uploaded after the network is restored.

[0047] Specifically, the face capture camera is a high-definition surveillance smart camera that can track, identify, and intelligently magnify faces. It supports detection, tracking, and capturing of moving faces. As long as they enter the surveillance range, they will capture and record the faces and output the optimal face capture at the same time. It also supports Micro SD / Micro SDHC / Micro SDXC cards, and supports local video storage and continued transmission when the network is disconnected.

[0048] S22. The server receives the uploaded captured face photo and obtains the nearest fishing boat position information based on the capture time, which is used for subsequent correlation analysis to determine whether the crew is actually operating at sea following the fishing boat when the photo was captured.

[0049] S23. The server calls the image comparison and search interface to retrieve the crew's identity information and certificate information, and establishes a list of crew information on board with the face of the captured photo as the primary key; when multiple photos of the crew are captured, the photo with the farthest position from the departure fishing port after the associated ship position is used first.

[0050] Specifically, the face image retrieval technology is used: a face picture is searched in the face database, and the most similar one or more faces and the corresponding similarity are returned; if there are M faces in a picture, it supports searching all faces in the face database at one time, and returns the user and similarity corresponding to each face; it supports using a face picture to compare with multiple photos of a specified user in the face database, and returns the similarity score with the user. The supported face comparison threshold setting range is. Real-time verification of the face's posture angle, occlusion, clarity, and lighting conditions to ensure that the output image meets the quality standards.

[0051] S24. The server associates the identified crew information with certificates and marks the official crew members and ordinary crew members. If there are faces that are not associated with crew certificates, they will be marked and an early warning will be generated that "the ship is suspected to have uncertified crew members operating at sea."

[0052] S3. Before fishing boats enter the sea or port: conduct business analysis and generate early warning information in real time.

[0053] Since fishing boats may not have network access when at sea, the system generates a real-time warning when the fishing boat has network access. If there is no network access, the system will upload historical snapshots and conduct business analysis when the mobile communication network (such as 4G / 5G network) is restored before entering the port. The specific steps include:

[0054] S31.According to the captain of the fishing vessel and the total power of the main engine, obtain the staffing standards for marine fishing vessels.

[0055] S32. Analyze whether the fishing vessel crew is fully staffed according to the marine fishery position crew staffing standards. When the crew certificate contains multiple positions such as driving and engineering, only one position can be used for standard matching, but all positions can be looped in turn to perform matching calculations. All crew members are compared with the standards according to their positions and marked after meeting the requirements. If all personnel and all positions are traversed in turn and still do not meet the standards, they will be marked and a warning of "the ship is suspected to be short of marine position crew staffing" will be generated.

[0056] S33. Compare the automatically generated list of crew information on board with the personnel reported by the captain when leaving the port, analyze whether the captured personnel are consistent with the reported personnel, and when there is an inconsistency, mark it and generate an early warning that "the ship is suspected to have crew members reported when leaving the port who are inconsistent with the actual crew members."

[0057] S34. After all non-repetitive facial information captured during the voyage is recognized, the total number of people is used to determine the number of crew members on the fishing vessel. The number of people is compared with the approved number of people inspected by the ship to analyze whether there is overcrowding. If there is overcrowding, it will be marked and a warning will be generated that "the number of crew members on board the ship is suspected to exceed the approved number."

[0058] S4. When a fishing boat enters the port: After identifying the fishing boat entering the port, a reminder will be given based on the situation of the fishing boat. If there is an abnormal warning, an onboard inspection will be arranged to check the actual situation. The specific steps include:

[0059] S41. When a fishing boat enters the port, it is identified through electronic fences or port videos, and a reminder of the fishing boat entering the port is generated, marking the end of the voyage.

[0060] S42. Check whether there are any abnormal marks on the fishing vessels entering the port on this voyage and whether relevant warning information has been generated.

[0061] S43. If the above situation information exists, the management personnel will arrange boarding inspection, and confirm, track and close the warning information based on the inspection results; if the above situation information does not exist, it means that there is no abnormal crew configuration on this voyage.

[0062] Specifically, spatial computing technology is used to identify fishing boats leaving the port in step S13 and entering the port in step S41, combined with electronic fences and spatial computing functions and memory spatial indexes to achieve high-performance, real-time electronic fence operations, provide a standard space-time processing framework for large-scale mobile objects, and realize the "online" application of spatial information in the field of fishing vessel supervision.

[0063] The boundary of the fishing port is defined by a set of coordinate points and stored in a spatial database for subsequent boundary calculation. The location data of fishing boats is collected in real time through Beidou and AIS equipment. Real-time data stream processing is performed through a real-time computing engine. For each fishing boat location data, the spatial computing function of the NoSQL library that supports geospatial computing, that is, the polygonal judgment algorithm, is used to determine whether the fishing boat is located within the electronic fence, and the entry or exit of the fishing boat is identified based on the association between the upper and lower points. The specific method is as follows:

[0064] Definition: The trajectory of a fishing boat is a spatiotemporal sequence of the position of a fishing boat within a period of time, which is described as: T = (P0, P1, ..., P n ), where n is the number of fishing boat track points collected during this period; P represents the track point of the fishing boat track, and the track point includes but is not limited to longitude, latitude and time; the system determines P in turn through the spatial algorithm n Is it within the electronic fence of all fishing ports? If it is identified in any fishing port, the calculation ends and P is associated n-1 ; such as P n In Hong Kong, P n-1If it is not in port, it is identified as a fishing boat entering the port; n Not in Hong Kong, P n-1 If in port, it is identified as a fishing vessel leaving the port.

[0065] The present invention also uses big data technology: a distributed service suitable for large-scale parallel batch processing jobs, which can support a large number of concurrent jobs. Batch computing is non-real-time and has high latency, and the data is generally static data. It is used in scenarios where real-time requirements are not high and offline computing is required, such as data analysis, offline reporting, etc. The batch computing tasks are generally completed at one time.

[0066] The present invention also adopts microservice architecture: microservice architecture is an architectural pattern that divides a single application into a group of small services, which coordinate and cooperate with each other and communicate with each other using lightweight communication mechanisms. Each service of the microservice is built around a specific business and can be deployed independently.

[0067] When the present invention is used, first, a fishery crew information database is constructed according to the fishery crew certification data; then, before the fishing boat leaves the port, the captain fills in the crew members leaving the port; when the fishing boat is operating at sea, the crew members' face photos are captured from time to time by the ship-borne camera, and uploaded to the service end through the marine broadband, the service end receives the uploaded captured photos, identifies the identity information, obtains the nearest fishing boat position information according to the capture time, and analyzes the crew members' deployment situation at sea according to the crew information and the position report; when there is an abnormal situation, the fishing boat is marked and an early warning message is generated; when the fishing boat enters the port, the fishing boat with abnormal early warning can be reminded according to the management requirements and arranged for boarding inspection.

[0068] By collecting and modeling the above information, we can automatically analyze the actual situation of the crew of a fishing vessel when operating at sea, prevent false reporting, misreporting, inconsistency between reported crew and actual crew, etc.; this automated method can also reduce the captain's reporting work when leaving the port, and enhance the management department's ability to analyze the seaworthiness of fishing vessel crew and its efficiency in handling them.

[0069] The present invention can be integrated as a module of a general system, or embedded in a terminal for independent use of functions. It provides a more convenient way to optimize and reduce the manual reporting work required during fishermen's production operations, form an innovation in the fishing vessel management mechanism, improve management efficiency, and promote the modernization of the fishery governance system and governance capabilities.

Claims

1. An automated analysis method for crew manning based on video capture and fishing vessel positioning, characterized in that: The following steps are involved: S1. Before the fishing vessel leaves the port: including daily construction of the fishery crew information database, the captain filling out the port departure report before leaving the port, and identifying the fishing vessel leaving the port; S2. When the fishing boat is at sea: When the fishing boat leaves the fishing port, the crew information is automatically captured by the video equipment and uploaded to the server. The server analyzes the crew and generates a list of crew information on board; S3. Before fishing boats are at sea or in port: conduct business analysis and generate early warning information in real time; S4. When a fishing boat enters the port: After identifying the fishing boat entering the port, a reminder will be given based on the situation of the fishing boat. If there is an abnormal warning, an onboard inspection will be arranged to check the actual situation.

2. The method for automatically analyzing crew manning conditions based on video capture and fishing vessel positioning according to claim 1, characterized in that: The step S1 specifically includes the following steps: S11. During the daily operation of the business system, the crew's photos, basic information, and certificate information are collected through the fishery crew certification data to build a fishery crew information database to achieve "one file for each crew member"; S12. Before the fishing vessel leaves the port, the captain can bind the crew members on board in advance, or bind them when filling in the report; select the fishing port, departure time, and crew members on board; the captain can directly use the crew information filled in or identified last time, or can use the crew information automatically captured and generated by the system as the crew list to be filled in; S13. The system identifies the fishing boat leaving the port through the fishing boat's position and electronic fence, and by judging the latest and last reported positions, and marks the fishing boat as leaving the port.

3. The method for automatically analyzing crew manning conditions based on video capture and fishing vessel positioning according to claim 1 is characterized in that: The step S2 specifically includes the following steps: S21. The system marks the fishing boat as being at sea by its position. When the fishing boat is operating at sea, the onboard cameras at different parts of the fishing boat periodically capture the facial photos of the crew members on board and upload them to the server via the marine broadband. When there is no network signal, the photos are cached and uploaded after the network is restored. S22. The server receives the uploaded captured face photo and obtains the nearest fishing boat position information according to the capture time, which is used for subsequent correlation analysis to determine whether the crew is actually operating at sea following the fishing boat when the photo was captured; S23. The server calls the image comparison and search interface to retrieve the crew's identity information and certificate information, and uses the captured photo's face as the primary key to create a list of crew information on board; when multiple photos of a crew are captured, the photo with the furthest position from the departure fishing port after the associated ship position is used first; S24. The server associates the identified crew information with certificates and marks the official crew members and ordinary crew members. If there are faces that are not associated with crew certificates, they will be marked and an early warning of "the ship is suspected to have uncertified crew members operating at sea" will be generated.

4. The method for automatically analyzing crew manning conditions based on video capture and fishing vessel positioning according to claim 1, characterized in that: In step S3, since there may be no network when the fishing boat is at sea, the system generates a real-time warning when the fishing boat has a network; if there is no network, when the mobile communication network is restored before entering the port, historical snapshots are uploaded and business analysis is carried out.

5. The method for automatically analyzing crew manning conditions based on video capture and fishing vessel positioning according to claim 1 is characterized in that: The step S3 specifically comprises the following steps: S31.According to the captain of the fishing vessel and the total power of the main engine, obtain the standard for the manning of marine fishing vessels; S32. Analyze whether the fishing vessel crew is fully staffed according to the marine fishery position crew staffing standards. When the crew certificate contains multiple positions such as driving and engine, only one position can be used for standard matching, but all positions can be cycled in sequence for matching calculation. All crew members are compared with the standards according to their positions and marked after meeting the requirements; if all personnel and all positions are traversed in sequence and still do not meet the standards, they will be marked and an early warning of "the ship is suspected of lacking marine position crew staffing" will be generated; S33. Compare the automatically generated list of crew information on board with the personnel reported by the captain when leaving the port, analyze whether the captured personnel are consistent with the reported personnel, and if there is any inconsistency, mark it and generate an early warning that "the reported crew of the ship is suspected to be inconsistent with the actual crew"; S34. After all non-repetitive facial information captured during the voyage is recognized, the total number of people is used to determine the number of crew members on the fishing vessel. The number of people is compared with the approved number of people inspected by the ship to analyze whether there is overcrowding. If there is overcrowding, it will be marked and a warning will be generated that "the number of crew members on board the ship is suspected to exceed the approved number." 6. The method for automated analysis of crew manning conditions based on video capture and fishing vessel positioning according to claim 1, characterized in that: The step S4 specifically comprises the following steps: S41. When a fishing boat enters the port, it is identified through electronic fences or port videos, and a reminder of the fishing boat entering the port is generated, and the end of the voyage is marked; S42. Check whether there are any abnormal marks on the fishing vessels entering the port on this voyage and whether relevant warning information is generated; S43. If the above situation information exists, the management personnel will arrange boarding inspection, and confirm, track and close the warning information based on the inspection results; if the above situation information does not exist, it means that there is no abnormal crew configuration on this voyage.

7. The method for automatically analyzing crew manning conditions based on video capture and fishing vessel positioning according to claim 1, characterized in that: The specific method of identifying the departure and arrival of fishing boats is as follows: the fishing boat trajectory is defined as a spatiotemporal sequence of the position of a fishing boat within a period of time, which is described as: T = (P0, P1, ..., P n ), where n is the number of fishing boat track points collected during this period; P represents the track point of the fishing boat track, and the track point includes but is not limited to longitude, latitude and time; the system determines P in turn through the spatial algorithm n Is it within the electronic fence of all fishing ports? If it is identified in any fishing port, the calculation ends and P is associated n-1 ; such as P n In Hong Kong, P n-1 If it is not in port, it is identified as a fishing boat entering the port; if P n Not in Hong Kong, P n-1 If in port, it is identified as a fishing vessel leaving the port.

Images