Ship safety monitoring method, device, equipment, medium and program product

Abstract

The invention relates to the technical field of ship systems, and discloses a ship safety monitoring method, device and equipment, a medium and a program product. The method comprises the following steps: obtaining a ship monitoring video, and carrying out window segmentation on the ship monitoring video to obtain a plurality of video clips; obtaining a mean value score and a standard deviation score corresponding to each video clip by calculating the similarity degree between the adjacent video clips, and performing clustering analysis on the plurality of video clips according to the mean value score and the standard deviation score to obtain a video clustering result; and according to a video clustering result, if it is detected that the target video clip belongs to a scene change class, carrying out ship safety alarm. According to the scheme of the embodiment, ship safety early warning is realized based on dynamic scene change detection, the labor cost and the time cost can be reduced, missing report and false report of ship safety abnormity can be avoided, and the reliability of ship safety monitoring can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of ship systems, and in particular to a ship safety monitoring method, device, equipment, medium and program product. Background Art

[0002] Unmanned boats can be used for rescue, detection and other activities in various complex marine environments. In the process of performing the above activities, due to the unmanned characteristics of unmanned boats, the safety protection of unmanned boats is of great significance for the unmanned boats to complete their operating tasks efficiently and orderly. The main issues involved are the navigation safety of unmanned boats and the safety protection against fires, flooding, etc. in the cabin.

[0003] At present, the existing safety monitoring methods for unmanned boats usually adopt the method of manual remote monitoring, that is, the monitoring personnel remotely view the monitoring video to determine whether the unmanned boat is in a safe state. However, this method has high labor and time costs, and there may be problems such as untimely detection of abnormalities, resulting in poor reliability of safety monitoring. Summary of the invention

[0004] The present invention provides a ship safety monitoring method, device, equipment, medium and program product, which can reduce labor costs and time costs, avoid underreporting and false reporting of ship safety anomalies, and improve the reliability of ship safety monitoring.

[0005] According to one aspect of the present invention, there is provided a method for monitoring ship safety, comprising:

[0006] Acquire a ship monitoring video, and perform window segmentation on the ship monitoring video to obtain multiple video clips;

[0007] By calculating the similarity between adjacent video clips, a mean score and a standard deviation score corresponding to each of the video clips are obtained, and cluster analysis is performed on the multiple video clips according to the mean score and the standard deviation score to obtain a video clustering result;

[0008] According to the video clustering result, if it is detected that the target video segment belongs to the scene change class, a ship safety alarm is issued.

[0009] According to another aspect of the present invention, there is provided a device for monitoring ship safety, comprising:

[0010] A video clip acquisition module is used to acquire a ship monitoring video and divide the ship monitoring video into windows to acquire multiple video clips;

[0011] A video segment clustering module, configured to obtain a mean score and a standard deviation score corresponding to each of the video segments by calculating the similarity between adjacent video segments, and to perform cluster analysis on the multiple video segments according to the mean score and the standard deviation score to obtain a video clustering result;

[0012] The ship safety alarm module is used to issue a ship safety alarm if it is detected that the target video segment belongs to the scene change category according to the video clustering result.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the ship safety monitoring method described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program is used to enable a processor to implement the ship safety monitoring method described in any embodiment of the present invention when executed.

[0018] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the ship safety monitoring method according to any embodiment of the present invention is implemented.

[0019] The technical solution of the embodiment of the present invention obtains a ship monitoring video and divides the ship monitoring video into windows to obtain multiple video clips; obtains the mean score and standard deviation score corresponding to each video clip by calculating the similarity between adjacent video clips, and performs cluster analysis on the multiple video clips according to the mean score and the standard deviation score to obtain video clustering results; according to the video clustering results, if it is detected that the target video clip belongs to the scene change class, a ship safety alarm is issued; by realizing ship safety early warning based on dynamic scene change detection, labor costs and time costs can be reduced, underreporting and false alarms of ship safety anomalies can be avoided, and the reliability of ship safety monitoring can be improved.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a flow chart of a method for monitoring ship safety provided according to Embodiment 1 of the present invention;

[0023] Figure 2 is a flow chart of another method for monitoring ship safety provided according to Embodiment 1 of the present invention;

[0024] Figure 3 is a calculation flow chart of similarity scores between projection feature graphs provided according to the first embodiment of the present invention;

[0025] Figure 4 is a schematic diagram of the structure of a ship safety monitoring device provided according to Embodiment 2 of the present invention;

[0026] Figure 5 It is a structural schematic diagram of an electronic device for implementing the ship safety monitoring method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", "target", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Embodiment 1

[0030] Figure 1 A flow chart of a ship safety monitoring method is provided for the first embodiment of the present invention. This embodiment is applicable to the situation of safety monitoring and early warning of unmanned boats. The method can be executed by a ship safety monitoring device. The ship safety monitoring device can be implemented in the form of hardware and / or software. Typically, the ship safety monitoring device can be configured in an electronic device, such as a computer device or a server. Figure 1 As shown, the method includes:

[0031] S110: Obtain a ship monitoring video, and divide the ship monitoring video into windows to obtain multiple video clips.

[0032] The ship may be an unmanned ship such as an unmanned boat. In this embodiment, a group of RGBD (Red, Green, Blue and Depth) cameras may be pre-installed on the ship to perform all-round monitoring of the surrounding external environment, deck and internal cabin of the unmanned boat, so as to acquire the ship monitoring video.

[0033] Specifically, the ship monitoring video can be acquired by a pre-deployed camera, and the ship monitoring video can be segmented into multiple video clips of equal duration using a preset sliding window method. The duration of the video clip is shorter than the duration of the ship monitoring video, and the video clips can overlap. For example, the sliding step size of the sliding window can be set to be smaller than the window width, so that the segmented video clips overlap.

[0034] S120, obtaining a mean score and a standard deviation score corresponding to each video segment by calculating the similarity between adjacent video segments, and performing cluster analysis on the multiple video segments according to the mean score and the standard deviation score to obtain a video clustering result.

[0035] In this embodiment, for each video segment, the previous video segment or the next video segment can be used as its corresponding adjacent video segment, and the similarity between each video segment and the corresponding adjacent video segment is calculated. Specifically, for the current video segment, first, each frame image of the current video segment and the corresponding adjacent video segment can be extracted respectively, and a one-to-one correspondence between each video frame image in the current video segment and each video frame image in the adjacent video segment is established in accordance with the extraction order; for example, for the first video frame image of the current video segment, it can be determined that the first video frame image of the adjacent video segment corresponds to it, and a correspondence between the two video frame images can be established. Then, a preset image similarity evaluation method can be used to calculate the similarity score between the two video frame images in each corresponding relationship. The type of image similarity evaluation method may not be specifically limited in this embodiment. Finally, the mean and standard deviation of all similarity scores can be calculated respectively to obtain the mean score and standard deviation score corresponding to the current video segment. In this way, the mean score and standard deviation score corresponding to each video segment can be obtained.

[0036] After obtaining the mean score and standard deviation score corresponding to each video clip, a clustering algorithm can be used to detect whether the ship scene has changed according to the mean score and standard deviation score corresponding to each video clip. Among them, the larger the mean score, the higher the similarity, and the smaller the standard deviation score, the higher the similarity. Specifically, a k-means clustering algorithm can be used to cluster each video clip according to the mean score and standard deviation score corresponding to each video clip to obtain a video clustering result. Typically, the k value can be set to 2. The video clustering result can include the center point of each cluster (each point corresponds to a video clip), and the cluster label corresponding to each video clip.

[0037] S130: According to the video clustering result, if it is detected that the target video segment belongs to a scene change class, a ship safety alarm is issued.

[0038] It should be noted that, in general, the monitoring scene of the unmanned boat is relatively stable. If a scene change occurs, it means that there is a large deviation from the design scene and there may be an abnormality. Therefore, after obtaining the video clustering results, if it is detected that the clustering label corresponding to a certain video clip is a scene change class, the detected video clip can be determined as the target video clip, that is, the video clip where the scene change has occurred. At this time, it can be determined that the monitoring scene has changed, and the ship safety alarm can be automatically issued. For example, the early warning system can be activated and sent to other systems on the ship through standard messages. This embodiment does not specifically limit the form of the ship safety alarm.

[0039] The technical solution of the embodiment of the present invention obtains a ship monitoring video and divides the ship monitoring video into windows to obtain multiple video clips; obtains the mean score and standard deviation score corresponding to each video clip by calculating the similarity between adjacent video clips, and performs cluster analysis on the multiple video clips according to the mean score and the standard deviation score to obtain video clustering results; according to the video clustering results, if it is detected that the target video clip belongs to the scene change class, a ship safety alarm is issued; by realizing ship safety early warning based on dynamic scene change detection, labor costs and time costs can be reduced, underreporting and false alarms of ship safety anomalies can be avoided, and the reliability of ship safety monitoring can be improved.

[0040] In an optional implementation of this embodiment, calculating the similarity between adjacent video clips to obtain the mean score and the standard deviation score corresponding to each of the video clips may include:

[0041] Perform frame extraction processing on each of the video clips to obtain a plurality of video frame images corresponding to each of the video clips;

[0042] By calculating the similarity between a plurality of video frame images corresponding to adjacent video clips, the mean score and the standard deviation score corresponding to each of the video clips are obtained.

[0043] In this embodiment, in order to improve the efficiency of similarity evaluation and reduce the amount of calculation, for each video clip, a frame image can be extracted every S frames to obtain multiple video frame images corresponding to each video clip. Among them, S can be set adaptively according to the task scenario. Then, according to the extraction order of the video frame images, a one-to-one correspondence between the video frame images corresponding to each video clip and the video frame images corresponding to the adjacent video clips can be established. In this way, multiple corresponding relationships corresponding to each video clip can be established. Finally, for each video clip, the similarity score between the two video frame images in each corresponding relationship can be calculated, and the mean and standard deviation of the similarity scores of each corresponding relationship can be calculated to obtain the mean score and standard deviation score corresponding to each video clip.

[0044] Optionally, calculating the similarity between a plurality of video frame images corresponding to adjacent video segments to obtain a mean score and a standard deviation score corresponding to each of the video segments may include:

[0045] Acquire a current video frame image corresponding to a current video segment, and acquire an associated video frame image corresponding to the current video frame image from a plurality of video frame images corresponding to video segments adjacent to the current video segment;

[0046] Calculating the similarity between the current video frame image and the associated video frame image to obtain a similarity score corresponding to the current video frame image, and obtaining a similarity score corresponding to each video frame image corresponding to the current video segment;

[0047] The mean of the similarity scores corresponding to the video frame images is calculated to obtain the mean score corresponding to the current video clip, and the standard deviation of the similarity scores corresponding to the video frame images is calculated to obtain the standard deviation score corresponding to the current video clip.

[0048] Specifically, when calculating the mean score and standard deviation score corresponding to each video clip, first, for the current video frame image corresponding to the current video clip, a video frame image having the same frame extraction order as the current video frame image can be searched among multiple video frame images corresponding to adjacent video clips to be used as an associated video frame image; for example, for the i-th video frame image of the current video clip, the i-th video frame image of the adjacent video clip can be determined as an associated video frame image. In this way, multiple corresponding relationships consisting of video frame images and associated video frame images can be obtained.

[0049] Then, the similarity between the current video frame image and the associated video frame image can be evaluated to obtain a similarity score corresponding to the current video frame image. For example, the similarity between the two video frame images can be evaluated using a histogram method, a cosine similarity method, a hash algorithm, etc. Finally, the mean and standard deviation of each similarity score can be calculated to obtain a mean score and a standard deviation score corresponding to the current video clip.

[0050] Optionally, calculating the similarity between the current video frame image and the associated video frame image to obtain a similarity score corresponding to the current video frame image may include:

[0051] Perform feature projection on the current video frame image to obtain a current projection feature map, and perform feature projection on the associated video frame image to obtain an associated projection feature map;

[0052] A similarity score between the current projection feature map and the associated projection feature map is calculated as the similarity score corresponding to the current video frame image.

[0053] Specifically, when evaluating the similarity between two video frame images, first, a preset feature projection method can be used, such as a principal component analysis method, a linear discriminant analysis method, etc., to map the current video frame image and the associated video frame image into a low-dimensional (two-dimensional) current projection feature map and an associated projection feature map, respectively. Then, a scale-invariant feature transformation method, an accelerated robust feature method, etc. can be used to calculate the similarity score between the two projection feature maps as the similarity score corresponding to the current video frame image.

[0054] In a specific example, a pre-trained vector quantization variational autoencoder model can be used to perform feature projection on a video frame image to obtain a corresponding projection feature map. Specifically, a nonlinear change can be applied by the model encoder to convert the video frame image into a vector E(x), which is quantized by comparing its distance with a series of vectors pre-defined in the encoding table to obtain a projection feature map. The encoding table of appropriate quantization is initialized by uniform quantization, expressed as G~U(-1 / N e , 1 / N e ), N e Represents the size of the encoding table. The model objective loss function can be defined as L=L vq +L re , L vq represents the vector loss function, L re represents the reconstruction loss function, which is defined as: .

[0055] where sg[·] represents the stopping gradient term whose partial derivative is set to 0, e represents the latent vector, β represents the hyperparameter used for tuning, D(e) represents the decoder output, and x represents the video frame image.

[0056] The advantage of the above setting is that it can reduce the amount of calculation for similarity evaluation and improve the efficiency of similarity evaluation.

[0057] In a specific implementation of this embodiment, the process of the ship safety monitoring method can be as follows: Figure 2As shown. First, obtain the ship monitoring video, and divide the ship monitoring video into windows to obtain multiple video clips; then, extract each video clip once every S frames to obtain multiple video frame images corresponding to each video clip; then, perform feature projection on each video frame image to obtain the corresponding projection feature map, that is, the feature matrix; further, calculate the similarity score between each projection feature map and the corresponding associated projection feature map as the similarity score corresponding to the corresponding video frame image; finally, calculate the mean and standard deviation of the similarity scores corresponding to each video frame image contained in each video clip to obtain the mean score and standard deviation score corresponding to each video clip, and perform cluster analysis on all video clips according to the mean score and standard deviation score to determine whether the scene has changed.

[0058] Optionally, calculating a similarity score between the current projection feature map and the associated projection feature map may include:

[0059] Divide the current projection feature map into cells to obtain a plurality of first cell features, and flatten each of the first cell features to obtain a first feature vector sequence corresponding to each of the first cell features;

[0060] Divide the associated projection feature map into cells to obtain multiple second cell features, and flatten each of the second cell features to obtain a second feature vector sequence corresponding to each of the second cell features;

[0061] According to the first feature vector sequence corresponding to each of the first cell features and the second feature vector sequence corresponding to each of the second cell features, if it is detected that there are overlapping feature vectors in the current first cell features, then the number of the overlapping feature vectors is obtained;

[0062] If the number of overlapping feature vectors is greater than a preset threshold, the current first cell feature is determined as a matching cell feature, and the ratio between the number of matching cell features and the number of the first cell features is calculated as a similarity score between the current projection feature map and the associated projection feature map.

[0063] In a specific example, the calculation process of the similarity score between projected feature maps can be as follows Figure 3Specifically, first, for the current projection feature map and the associated projection feature map, the two-dimensional projection feature map is divided into N cells of the same size to obtain multiple first cell features and second cell features. Then, each first cell feature and second cell feature can be flattened (feature dimension reduction) to obtain a first feature vector sequence corresponding to each first cell feature and a second feature vector sequence corresponding to each second cell feature.

[0064] Furthermore, the similarity of the cell features in the same position can be evaluated. Specifically, the number of occurrences of each feature vector in each feature vector sequence can be counted, and the feature vectors that appear in both feature vector sequences and have the same number of occurrences in the two feature vector sequences are determined to be overlapping feature vectors. If the overlapping feature vectors are successfully detected in the current first cell feature, the number of overlapping feature vectors is counted. Afterwards, the number of overlapping feature vectors can be compared with a preset threshold to evaluate the similarity between the two cell features. If the number of overlapping feature vectors is greater than the preset threshold, it means that the two cell features are similar, and the two cell features can be determined as matching cell features. Finally, the ratio between the number of matching cell features and the number of first cell features can be calculated as the similarity score between the two projected feature maps.

[0065] For example, the cell score I corresponding to the matching cell feature can be set to 1, and the cell score I corresponding to other cell features can be set to 0, and then based on the formula The similarity score sim between the two projected feature maps is calculated.

[0066] Optionally, detecting that there are overlapping feature vectors in the current first cell features according to the first feature vector sequence corresponding to each of the first cell features and the second feature vector sequence corresponding to each of the second cell features may include:

[0067] According to the position of the current first cell feature in the current projection feature map, obtain the target second cell feature that matches the current first cell feature;

[0068] Acquire multiple target first feature vectors according to the number of occurrences of each first feature vector in the first feature vector sequence corresponding to the current first cell feature, and acquire multiple target second feature vectors according to the number of occurrences of each second feature vector in the second feature vector sequence corresponding to the target second cell feature;

[0069] If it is detected that the current target first feature vector is equal to a certain target second feature vector, and the corresponding number of occurrences is equal, the current target first feature vector is determined as an overlapping feature vector.

[0070] Specifically, when determining overlapping feature vectors, first, according to the position (row, column) of the current first cell feature in the current projection feature map, the second cell feature at the same position can be found from the associated projection feature map as the target second cell feature. Then, all first feature vectors can be sorted in descending order according to the number of occurrences of each first feature vector in the first feature vector sequence corresponding to the current first cell feature, and a specified number of first feature vectors can be selected from the first feature vector after descending order as the target first feature vector. Similarly, the above method can be used to screen the same number of target second feature vectors in all second feature vectors. Finally, the target first feature vectors can be judged one by one. If it is detected that a certain target first feature vector is equal to a certain target second feature vector, and the corresponding number of occurrences is also equal, that is, it appears in two feature vector sequences at the same time and the number of occurrences is the same, then the detected target first feature vector can be determined as an overlapping feature vector.

[0071] Embodiment 2

[0072] Figure 4 This is a schematic diagram of the structure of a ship safety monitoring device provided in Embodiment 2 of the present invention. Figure 4 As shown, the device includes: a video segment acquisition module 210, a video segment clustering module 220 and a ship safety alarm module 230; wherein,

[0073] The video segment acquisition module 210 is used to acquire a ship monitoring video and perform window segmentation on the ship monitoring video to acquire multiple video segments;

[0074] The video segment clustering module 220 is used to obtain the mean score and the standard deviation score corresponding to each of the video segments by calculating the similarity between adjacent video segments, and perform cluster analysis on the multiple video segments according to the mean score and the standard deviation score to obtain a video clustering result;

[0075] The ship safety alarm module 230 is used to issue a ship safety alarm if it is detected that the target video segment belongs to the scene change category according to the video clustering result.

[0076] The technical solution of the embodiment of the present invention obtains a ship monitoring video and divides the ship monitoring video into windows to obtain multiple video clips; obtains the mean score and standard deviation score corresponding to each video clip by calculating the similarity between adjacent video clips, and performs cluster analysis on the multiple video clips according to the mean score and the standard deviation score to obtain video clustering results; according to the video clustering results, if it is detected that the target video clip belongs to the scene change class, a ship safety alarm is issued; by realizing ship safety early warning based on dynamic scene change detection, labor costs and time costs can be reduced, underreporting and false alarms of ship safety anomalies can be avoided, and the reliability of ship safety monitoring can be improved.

[0077] Optionally, the video segment clustering module 220 is specifically configured to perform frame extraction processing on each of the video segments to obtain a plurality of video frame images corresponding to each of the video segments;

[0078] By calculating the similarity between a plurality of video frame images corresponding to adjacent video clips, the mean score and the standard deviation score corresponding to each of the video clips are obtained.

[0079] Optionally, the video segment clustering module 220 is specifically configured to obtain a current video frame image corresponding to a current video segment, and obtain an associated video frame image corresponding to the current video frame image from a plurality of video frame images corresponding to video segments adjacent to the current video segment;

[0080] Calculating the similarity between the current video frame image and the associated video frame image to obtain a similarity score corresponding to the current video frame image, and obtaining a similarity score corresponding to each video frame image corresponding to the current video segment;

[0081] The mean of the similarity scores corresponding to the video frame images is calculated to obtain the mean score corresponding to the current video clip, and the standard deviation of the similarity scores corresponding to the video frame images is calculated to obtain the standard deviation score corresponding to the current video clip.

[0082] Optionally, the video segment clustering module 220 is specifically configured to perform feature projection on the current video frame image to obtain a current projection feature map, and perform feature projection on the associated video frame image to obtain an associated projection feature map;

[0083] A similarity score between the current projection feature map and the associated projection feature map is calculated as the similarity score corresponding to the current video frame image.

[0084] Optionally, the video clip clustering module 220 is specifically used to divide the current projection feature map into cells, obtain multiple first cell features, and flatten each of the first cell features to obtain a first feature vector sequence corresponding to each of the first cell features;

[0085] Divide the associated projection feature map into cells to obtain multiple second cell features, and flatten each of the second cell features to obtain a second feature vector sequence corresponding to each of the second cell features;

[0086] According to the first feature vector sequence corresponding to each of the first cell features and the second feature vector sequence corresponding to each of the second cell features, if it is detected that there are overlapping feature vectors in the current first cell features, then the number of the overlapping feature vectors is obtained;

[0087] If the number of overlapping feature vectors is greater than a preset threshold, the current first cell feature is determined as a matching cell feature, and the ratio between the number of matching cell features and the number of the first cell features is calculated as a similarity score between the current projection feature map and the associated projection feature map.

[0088] Optionally, the video segment clustering module 220 is specifically configured to obtain a target second cell feature matched with the current first cell feature according to a position of the current first cell feature in the current projection feature map;

[0089] Acquire multiple target first feature vectors according to the number of occurrences of each first feature vector in the first feature vector sequence corresponding to the current first cell feature, and acquire multiple target second feature vectors according to the number of occurrences of each second feature vector in the second feature vector sequence corresponding to the target second cell feature;

[0090] If it is detected that the current target first feature vector is equal to a certain target second feature vector, and the corresponding number of occurrences is equal, the current target first feature vector is determined as an overlapping feature vector.

[0091] The ship safety monitoring device provided in the embodiment of the present invention can execute the ship safety monitoring method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0092] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0093] Embodiment 3

[0094] Figure 5 A schematic diagram of the structure of an electronic device 30 that can be used to implement an embodiment of the present invention is shown. The electronic device 30 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 30 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0095] like Figure 5 As shown, the electronic device 30 includes at least one processor 31, and a memory connected to the at least one processor 31, such as a read-only memory (ROM) 32, a random access memory (RAM) 33, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 31 can perform various appropriate actions and processes according to the computer program stored in the read-only memory 32 or the computer program loaded from the storage unit 38 to the random access memory 33. In the RAM 33, various programs and data required for the operation of the electronic device 30 can also be stored. The processor 31, the ROM 32 and the RAM 33 are connected to each other through a bus 34. The input / output (I / O) interface 35 is also connected to the bus 34.

[0096] A number of components in the electronic device 30 are connected to the I / O interface 35, including: an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a disk, an optical disk, etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 39 allows the electronic device 30 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0097] The processor 31 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 31 include, but are not limited to, a central processing unit, a graphics processing unit, various special artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any appropriate processors, controllers, microcontrollers, etc. The processor 31 executes the various methods and processes described above, such as a method for monitoring ship safety.

[0098] In some embodiments, the ship safety monitoring method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 30 via the ROM 32 and / or the communication unit 39. When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the ship safety monitoring method described above may be performed. Alternatively, in other embodiments, the processor 31 may be configured to perform the ship safety monitoring method in any other appropriate manner (e.g., by means of firmware).

[0099] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays, application specific integrated circuits, application specific standard products, systems on a chip, load programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0100] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0101] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0102] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device 30 having: a display device (e.g., a cathode ray tube or a liquid crystal display) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device 30. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0103] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area networks, wide area networks, blockchain networks, and the Internet.

[0104] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other. The server may be a cloud server.

[0105] This embodiment may also include a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the ship safety monitoring method provided by any embodiment of the present invention.

[0106] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0107] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring ship safety, characterized in that: include: Acquire a ship monitoring video, and perform window segmentation on the ship monitoring video to obtain multiple video clips; By calculating the similarity between adjacent video clips, a mean score and a standard deviation score corresponding to each of the video clips are obtained, and cluster analysis is performed on the multiple video clips according to the mean score and the standard deviation score to obtain a video clustering result; According to the video clustering result, if it is detected that the target video segment belongs to the scene change class, a ship safety alarm is issued.

2. The method according to claim 1, characterized in that By calculating the similarity between adjacent video clips, the mean score and standard deviation score corresponding to each of the video clips are obtained, including: Perform frame extraction processing on each of the video clips to obtain a plurality of video frame images corresponding to each of the video clips; By calculating the similarity between a plurality of video frame images corresponding to adjacent video clips, the mean score and the standard deviation score corresponding to each of the video clips are obtained.

3. The method according to claim 2, characterized in that By calculating the similarity between multiple video frame images corresponding to adjacent video clips, the mean score and standard deviation score corresponding to each of the video clips are obtained, including: Acquire a current video frame image corresponding to a current video segment, and acquire an associated video frame image corresponding to the current video frame image from a plurality of video frame images corresponding to video segments adjacent to the current video segment; Calculating the similarity between the current video frame image and the associated video frame image to obtain a similarity score corresponding to the current video frame image, and obtaining a similarity score corresponding to each video frame image corresponding to the current video segment; The mean of the similarity scores corresponding to the video frame images is calculated to obtain the mean score corresponding to the current video clip, and the standard deviation of the similarity scores corresponding to the video frame images is calculated to obtain the standard deviation score corresponding to the current video clip.

4. The method according to claim 3, characterized in that Calculating the similarity between the current video frame image and the associated video frame image to obtain a similarity score corresponding to the current video frame image includes: Perform feature projection on the current video frame image to obtain a current projection feature map, and perform feature projection on the associated video frame image to obtain an associated projection feature map; A similarity score between the current projection feature map and the associated projection feature map is calculated as the similarity score corresponding to the current video frame image.

5. The method according to claim 4, characterized in that Calculating a similarity score between the current projection feature map and the associated projection feature map includes: Divide the current projection feature map into cells to obtain a plurality of first cell features, and flatten each of the first cell features to obtain a first feature vector sequence corresponding to each of the first cell features; Divide the associated projection feature map into cells to obtain multiple second cell features, and flatten each of the second cell features to obtain a second feature vector sequence corresponding to each of the second cell features; According to the first feature vector sequence corresponding to each of the first cell features and the second feature vector sequence corresponding to each of the second cell features, if it is detected that there are overlapping feature vectors in the current first cell features, then the number of the overlapping feature vectors is obtained; If the number of overlapping feature vectors is greater than a preset threshold, the current first cell feature is determined as a matching cell feature, and the ratio between the number of matching cell features and the number of the first cell features is calculated as a similarity score between the current projection feature map and the associated projection feature map.

6. The method according to claim 5, characterized in that According to the first feature vector sequence corresponding to each of the first cell features and the second feature vector sequence corresponding to each of the second cell features, detecting that there are overlapping feature vectors in the current first cell features includes: According to the position of the current first cell feature in the current projection feature map, obtain the target second cell feature that matches the current first cell feature; Acquire multiple target first feature vectors according to the number of occurrences of each first feature vector in the first feature vector sequence corresponding to the current first cell feature, and acquire multiple target second feature vectors according to the number of occurrences of each second feature vector in the second feature vector sequence corresponding to the target second cell feature; If it is detected that the current target first feature vector is equal to a certain target second feature vector, and the corresponding number of occurrences is equal, the current target first feature vector is determined as an overlapping feature vector.

7. A ship safety monitoring device, characterized in that: include: A video clip acquisition module is used to acquire a ship monitoring video and divide the ship monitoring video into windows to acquire multiple video clips; A video segment clustering module, configured to obtain a mean score and a standard deviation score corresponding to each of the video segments by calculating the similarity between adjacent video segments, and to perform cluster analysis on the multiple video segments according to the mean score and the standard deviation score to obtain a video clustering result; The ship safety alarm module is used to issue a ship safety alarm if it is detected that the target video segment belongs to the scene change category according to the video clustering result.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor, and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the ship safety monitoring method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the ship safety monitoring method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the ship safety monitoring method according to any one of claims 1 to 6.

Images