Monitoring Method, Device, Equipment, Medium and Program Product for Ship Safety

Abstract

The present invention relates to the technical field of ship systems, and discloses a method, device, equipment, medium and program product for ship safety monitoring. The method includes: obtaining a ship monitoring video, performing window segmentation on the ship monitoring video to obtain a plurality of video segments; calculating the similarity degree between adjacent video segments, obtaining the mean score and standard deviation score corresponding to each video segment, and performing clustering analysis on the plurality of video segments according to the mean score and standard deviation score to obtain a video clustering result; according to the video clustering result, if it is detected that the target video segment belongs to the scene change category, a ship safety alarm is given. The solution of this embodiment can realize ship safety early warning based on dynamic scene change detection, reduce labor costs and time costs, avoid missed reports and false alarms of ship safety anomalies, and improve the reliability of ship safety monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of ship systems, and particularly to a method, device, equipment, medium and program product for monitoring ship safety. Background Art

[0002] Unmanned boats can be used for activities such as rescue and detection in various complex marine environments. During the process of performing the above activities, due to the unattended feature, the safety protection of unmanned boats is of great significance for the efficient and orderly completion of operation tasks by unmanned boats, mainly involving the navigation safety of unmanned boats and the safety protection against fires, waterlogging, etc. in the cabin.

[0003] Currently, the existing safety monitoring methods for unmanned boats usually adopt the method of manual remote monitoring, that is, monitoring personnel remotely view the monitoring video to judge whether the unmanned boat is in a safe state. However, this method has high labor costs and time costs, and there may be problems such as untimely discovery of abnormalities, resulting in poor reliability of safety monitoring. Summary of the Invention

[0004] The present invention provides a method, device, equipment, medium and program product for monitoring ship safety, which can reduce labor costs and time costs, can avoid missed reports and false reports of ship safety abnormalities, and can improve the reliability of ship safety monitoring.

[0005] According to one aspect of the present invention, there is provided a method for monitoring ship safety, including:

[0006] Obtain a ship monitoring video, and perform window segmentation on the ship monitoring video to obtain a plurality of video segments;

[0007] By calculating the similarity degree between adjacent video segments, obtain the mean score and standard deviation score corresponding to each video segment, and perform clustering analysis on the plurality of video segments according to the mean score and the standard deviation score to obtain a video clustering result;

[0008] According to the video clustering result, if it is detected that the target video segment belongs to the scene change category, perform a ship safety alarm.

[0009] According to another aspect of the present invention, there is provided a device for monitoring ship safety, including:

[0010] A video segment acquisition module, configured to obtain a ship monitoring video, and perform window segmentation on the ship monitoring video to obtain a plurality of video segments;

[0011] A video clip clustering module, configured to calculate the similarity degree between adjacent video clips, obtain the mean score and the standard deviation score corresponding to each of the video clips, and perform clustering analysis on the multiple video clips according to the mean score and the standard deviation score to obtain a video clustering result;

[0012] A ship safety alarm module, configured to perform a ship safety alarm according to the video clustering result if it is detected that a target video clip belongs to the scene change category.

[0013] According to another aspect of the present invention, there is provided an electronic device, where the electronic device includes:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the ship safety monitoring method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, there is provided a computer-readable storage medium storing a computer program, and the computer program is used to implement the ship safety monitoring method according to any embodiment of the present invention when executed by a processor.

[0018] According to another aspect of the present invention, there is provided a computer program product including a computer program, and the computer program implements the ship safety monitoring method according to any embodiment of the present invention when executed by a processor.

[0019] The technical solution of the embodiment of the present invention obtains a ship monitoring video, performs window segmentation on the ship monitoring video to obtain multiple video clips; calculates the similarity degree between adjacent video clips, obtains the mean score and the standard deviation score corresponding to each video clip, and performs clustering analysis on the multiple video clips according to the mean score and the standard deviation score to obtain a video clustering result; according to the video clustering result, if it is detected that a target video clip belongs to the scene change category, a ship safety alarm is performed; by realizing ship safety early warning based on dynamic scene transformation detection, the labor cost and time cost can be reduced, the missed report and false report of ship safety anomalies can be avoided, and the reliability of ship safety monitoring can be improved.

[0020] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 is a flowchart of a method for monitoring ship safety provided in Embodiment 1 of the present invention;

[0023] Figure 2 is a flowchart of another method for monitoring ship safety provided in Embodiment 1 of the present invention;

[0024] Figure 3 is a flowchart of calculating the similarity score between projection feature maps provided in Embodiment 1 of the present invention;

[0025] Figure 4 is a schematic structural diagram of a device for monitoring ship safety provided in Embodiment 2 of the present invention;

[0026] Figure 5 is a schematic structural diagram of an electronic device for implementing the method for monitoring ship safety in the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0028] It should be noted that the terms "first", "second", "target", etc. in the specification and claims of the present invention and the above drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0029] Embodiment 1

[0030] Figure 1 The following is a flowchart of a method for monitoring the safety of a ship provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of monitoring the safety and giving early warnings to unmanned boats. This method can be executed by a monitoring device for ship safety, and the monitoring device for ship safety can be implemented in the form of hardware and / or software. Typically, the monitoring device for ship safety can be configured in an electronic device, such as a computer device or a server, etc. As Figure 1 shown, the method includes:

[0031] S110. Obtain a ship monitoring video, perform window segmentation on the ship monitoring video, and obtain a plurality of video segments.

[0032] Among them, the ship can be an unmanned boat or other unmanned ships. In this embodiment, a set of RGBD (Red, Green, Blue and Depth) cameras can be pre-installed on the ship to comprehensively monitor the surrounding external environment, deck and internal cabins of the unmanned boat, etc., so as to collect the ship monitoring video.

[0033] Specifically, the ship monitoring video can be collected through pre-deployed cameras, and the ship monitoring video can be segmented into a plurality of video segments with equal time lengths by using a preset sliding window method. Among them, the time length of the video segment is less than the time length of the ship monitoring video, and there may be an overlap between the video segments. For example, the sliding step length of the sliding window can be set to be less than the window width so that the segmented video segments overlap.

[0034] S120. By calculating the similarity degree between adjacent video segments, obtain the mean score and standard deviation score corresponding to each video segment, and perform clustering analysis on the plurality of video segments according to the mean score and the standard deviation score to obtain a video clustering result.

[0035] In this embodiment, for each video segment, its previous or next video segment can be used as its corresponding adjacent video segment, and the similarity between each video segment and its corresponding adjacent video segment can be calculated. Specifically, for the current video segment, first, each frame image of the current video segment and its corresponding adjacent video segment can be extracted respectively, and a one-to-one correspondence can be established between each video frame image in the current video segment and each video frame image in the adjacent video segment according to the extraction order; for example, for the first video frame image of the current video segment, the first video frame image of the adjacent video segment can be determined to correspond to it, and the correspondence between the two video frame images can be established. Then, a preset image similarity evaluation method can be used to calculate the similarity score between the two video frame images in each corresponding relationship. The type of the image similarity evaluation method in this embodiment can not be specifically limited. Finally, the mean and standard deviation of all similarity scores can be calculated respectively to obtain the mean score and standard deviation score corresponding to the current video segment. Thus, the mean score and standard deviation score corresponding to each video segment can be obtained.

[0036] After obtaining the mean score and standard deviation score corresponding to each video segment, the clustering algorithm can be used to detect whether the ship scene has changed according to the mean score and standard deviation score corresponding to each video segment. Among them, the larger the mean score, the higher the similarity, and the smaller the standard deviation score, the higher the similarity. Specifically, the k-means clustering algorithm can be used to perform clustering processing on each video segment according to the mean score and standard deviation score corresponding to each video segment to obtain the video clustering result. Typically, the value of k can be set to 2. The video clustering result can include the center point of each cluster (each point corresponds to a video segment), and the cluster label corresponding to each video segment.

[0037] S130. According to the video clustering result, if it is detected that the target video segment belongs to the scene change category, a ship safety alarm is issued.

[0038] It should be noted that generally, the monitoring scene of the unmanned boat is relatively stable. If the scene changes, it means that there is a large deviation from the designed scene, and there may be an abnormality. Thus, after obtaining the video clustering result, if it is detected that the cluster label corresponding to a certain video segment is the scene change category, the detected video segment can be determined as the target video segment, that is, the video segment with the scene change. At this time, it can be determined that the monitoring scene has changed, and a ship safety alarm is automatically issued. For example, the warning system can be started and sent to other systems on the ship through a standard message. The form of the ship safety alarm in this embodiment can not be specifically limited.

[0039] The technical solution of the embodiment of the present invention obtains a ship monitoring video, divides the ship monitoring video into windows to obtain multiple video segments; calculates the similarity between adjacent video segments, obtains the mean score and standard deviation score corresponding to each video segment, and performs clustering analysis on the multiple video segments according to the mean score and standard deviation score to obtain a video clustering result; according to the video clustering result, if it is detected that the target video segment belongs to the scene change category, a ship safety alarm is issued; by realizing ship safety early warning based on dynamic scene change detection, the labor cost and time cost can be reduced, the missed alarm and false alarm of ship safety anomalies can be avoided, and the reliability of ship safety monitoring can be improved.

[0040] In an alternative implementation manner of this embodiment, calculating the similarity between adjacent video segments and obtaining the mean score and standard deviation score corresponding to each video segment may include:

[0041] Perform frame extraction on each of the video segments to obtain multiple video frame images corresponding to each of the video segments;

[0042] Calculate the similarity between the multiple video frame images corresponding to adjacent video segments to obtain the mean score and standard deviation score corresponding to each video segment.

[0043] In this embodiment, in order to improve the similarity evaluation efficiency and reduce the calculation amount, for each video segment, one frame of image may be extracted every S frames to obtain multiple video frame images corresponding to each video segment. Wherein, S can be adaptively set according to the task scenario. Then, according to the extraction order of the video frame images, a one-to-one correspondence relationship can be established between the video frame images corresponding to each video segment and the video frame images corresponding to the adjacent video segments. Thus, multiple correspondence relationships corresponding to each video segment can be established. Finally, for each video segment, the similarity score between two video frame images in each corresponding relationship can be calculated, and the mean and standard deviation of the similarity scores of each corresponding relationship can be calculated to obtain the mean score and standard deviation score corresponding to each video segment.

[0044] Optionally, calculating the similarity between the multiple video frame images corresponding to adjacent video segments and obtaining the mean score and standard deviation score corresponding to each video segment may include:

[0045] Obtain the current video frame image corresponding to the current video segment, and obtain the associated video frame image corresponding to the current video frame image among the multiple video frame images corresponding to the video segment adjacent to the current video segment;

[0046] Calculate the similarity degree between the current video frame image and the associated video frame image to obtain the similarity score corresponding to the current video frame image, and obtain the similarity scores corresponding to the video frame images of the current video segment;

[0047] Calculate the mean value of the similarity scores corresponding to the video frame images to obtain the mean score corresponding to the current video segment, and calculate the standard deviation of the similarity scores corresponding to the video frame images to obtain the standard deviation score corresponding to the current video segment.

[0048] Specifically, when calculating the mean score and the standard deviation score corresponding to each video segment, first, for the current video frame image corresponding to the current video segment, among the multiple video frame images corresponding to the adjacent video segments, the video frame image with the same frame extraction order as the current video frame image can be found as the associated video frame image; for example, for the i-th video frame image of the current video segment, the i-th video frame image of the adjacent video segment can be determined as the associated video frame image. Thus, multiple corresponding relationships composed of video frame images and associated video frame images can be obtained.

[0049] Then, the similarity between the current video frame image and the associated video frame image can be evaluated to obtain the similarity score corresponding to the current video frame image. For example, methods such as the histogram method, the cosine similarity method, and the hash algorithm can be used to evaluate the similarity between two video frame images. Finally, the mean value and the standard deviation of the similarity scores can be calculated to obtain the mean score and the standard deviation score corresponding to the current video segment.

[0050] Optionally, calculating the similarity degree between the current video frame image and the associated video frame image to obtain the similarity score corresponding to the current video frame image may include:

[0051] Perform feature projection on the current video frame image to obtain the current projection feature map, and perform feature projection on the associated video frame image to obtain the associated projection feature map;

[0052] Calculate the similarity score between the current projection feature map and the associated projection feature map as the similarity score corresponding to the current video frame image.

[0053] Specifically, when evaluating the similarity between two video frame images, first, a preset feature projection method can be adopted, such as the principal component analysis method, the linear discriminant analysis method, etc., to map the current video frame image and the associated video frame image into a low-dimensional (two-dimensional) current projection feature map and associated projection feature map, respectively. Then, methods such as the scale-invariant feature transform method and the speeded-up robust features method can be used to calculate the similarity score between the two projection feature maps as the similarity score corresponding to the current video frame image.

[0054] In a specific example, a pre-trained vector quantization variational autoencoder model can be used to perform feature projection on the video frame image to obtain the corresponding projection feature map. Specifically, a nonlinear transformation can be applied through the model encoder to convert the video frame image into a vector E(x), and the vector is quantized by comparing its distance with a series of vectors predefined in the coding table to obtain the projection feature map. Among them, the coding table of the appropriate quantization is initialized by uniform quantization, denoted as G~U(-1 / N e , 1 / N e ), N e represents the coding table size. The model objective loss function can be defined as L = L vq +L re , L vq represents the vector loss function, and L re represents the reconstruction loss function, which are respectively defined as: .

[0055] Among them, sg[·] represents the stop gradient term, whose partial derivative is set to 0, e represents the latent vector, β represents the hyperparameter for tuning, D(e) represents the decoder output, and x represents the video frame image.

[0056] The above settings have the advantage of reducing the computational amount of similarity evaluation and improving the similarity evaluation efficiency.

[0057] In a specific implementation manner of this embodiment, the process of the ship safety monitoring method can be as Figure 2As shown below. First, obtain the ship monitoring video, perform window segmentation on the ship monitoring video to obtain multiple video segments; then, perform frame extraction on each video segment every S frames to obtain multiple video frame images corresponding to each video segment; after that, perform feature projection on each video frame image to obtain the corresponding projection feature map, that is, the feature matrix; further, calculate the similarity score between each projection feature map and the corresponding associated projection feature map as the similarity score corresponding to the corresponding video frame image; finally, calculate the mean and standard deviation of the similarity scores corresponding to each video frame image included in each video segment to obtain the mean score and standard deviation score corresponding to each video segment, and perform clustering analysis on all video segments based on the mean score and standard deviation score to determine whether the scene has changed.

[0058] Optionally, calculating the similarity score between the current projection feature map and the associated projection feature map may include:

[0059] Divide the current projection feature map into cells to obtain multiple first cell features, and flatten each of the first cell features to obtain a first feature vector sequence corresponding to each of the first cell features;

[0060] Divide the associated projection feature map into cells to obtain multiple second cell features, and flatten each of the second cell features to obtain a second feature vector sequence corresponding to each of the second cell features;

[0061] Based on the first feature vector sequence corresponding to each of the first cell features and the second feature vector sequence corresponding to each of the second cell features, if an overlapping feature vector is detected in the current first cell feature, obtain the number of the overlapping feature vectors;

[0062] If the number of the overlapping feature vectors is greater than a preset threshold, determine the current first cell feature as a matching cell feature, and calculate the ratio between the number of the matching cell features and the number of the first cell features as the similarity score between the current projection feature map and the associated projection feature map.

[0063] In a specific example, the calculation process of the similarity score between projection feature maps may be as Figure 3As shown. Specifically, first, for the current projection feature map and the associated projection feature map, the two-dimensional projection feature map is divided into N cells of the same size to obtain a plurality of first cell features and second cell features. Then, each first cell feature and second cell feature can be respectively flattened (feature dimensionality reduction) to obtain a first feature vector sequence corresponding to each first cell feature and a second feature vector sequence corresponding to each second cell feature.

[0064] Furthermore, similarity evaluation can be performed on cell features in the same position. Specifically, the occurrence times of each feature vector in each feature vector sequence can be counted, and the feature vectors that appear in both feature vector sequences and have the same occurrence times in both feature vector sequences are determined as overlapping feature vectors. If overlapping feature vectors are successfully detected in the current first cell feature, the number of overlapping feature vectors is counted. Then, the number of overlapping feature vectors can be compared with a preset threshold to evaluate the similarity degree between the two cell features. If the number of overlapping feature vectors is greater than the preset threshold, it indicates that the two cell features are similar, and the two cell features can be determined as matching cell features. Finally, the ratio between the number of matching cell features and the number of first cell features can be calculated as the similarity score between the two projection feature maps.

[0065] Exemplarily, the cell score I corresponding to the matching cell feature can be set to 1, and the cell score I corresponding to other cell features can be set to 0. Then, based on the formula the similarity score sim between the two projection feature maps is calculated.

[0066] Optionally, according to the first feature vector sequences corresponding to the first cell features and the second feature vector sequences corresponding to the second cell features, detecting that there are overlapping feature vectors in the current first cell feature may include:

[0067] According to the position of the current first cell feature in the current projection feature map, obtain the target second cell feature matched by the current first cell feature;

[0068] According to the occurrence times corresponding to each first feature vector in the first feature vector sequence corresponding to the current first cell feature, obtain a plurality of target first feature vectors, and according to the occurrence times corresponding to each second feature vector in the second feature vector sequence corresponding to the target second cell feature, obtain a plurality of target second feature vectors;

[0069] If it is detected that the current target first feature vector is equal to a certain target second feature vector and the corresponding occurrence times are equal, then the current target first feature vector is determined as an overlapping feature vector.

[0070] Specifically, when determining the overlapping feature vectors, first, according to the position (row, column) of the current first cell feature in the current projection feature map, the second cell feature at the same position can be found from the associated projection feature map as the target second cell feature. Then, all the first feature vectors can be sorted in descending order according to the number of occurrences of each first feature vector in the first feature vector sequence corresponding to the current first cell feature, and a specified number of first feature vectors can be selected from the beginning of the first feature vectors sorted in descending order as the target first feature vectors. Similarly, the above method can be used to screen the same number of target second feature vectors from all the second feature vectors. Finally, each target first feature vector can be judged one by one. If it is detected that a certain target first feature vector is equal to a certain target second feature vector and the corresponding number of occurrences is also equal, that is, they appear in both feature vector sequences at the same time and the number of occurrences is the same, then the detected target first feature vector can be determined as the overlapping feature vector.

[0071] Embodiment 2

[0072] Figure 4 It is a schematic structural diagram of a ship safety monitoring device provided in Embodiment 2 of the present invention. As Figure 4 shown, the device includes: a video clip acquisition module 210, a video clip clustering module 220, and a ship safety alarm module 230; wherein,

[0073] The video clip acquisition module 210 is configured to acquire a ship monitoring video, perform window segmentation on the ship monitoring video, and obtain a plurality of video clips;

[0074] The video clip clustering module 220 is configured to calculate the similarity degree between adjacent video clips, obtain the mean score and the standard deviation score corresponding to each video clip, and perform clustering analysis on the plurality of video clips according to the mean score and the standard deviation score to obtain a video clustering result;

[0075] The ship safety alarm module 230 is configured to perform a ship safety alarm according to the video clustering result if it is detected that the target video clip belongs to the scene change category.

[0076] The technical solution of the embodiment of the present invention obtains a ship monitoring video, performs window segmentation on the ship monitoring video to obtain multiple video segments; calculates the similarity degree between adjacent video segments, obtains the mean score and standard deviation score corresponding to each video segment, and performs clustering analysis on the multiple video segments according to the mean score and standard deviation score to obtain a video clustering result; according to the video clustering result, if it is detected that the target video segment belongs to the scene change category, a ship safety alarm is issued; by realizing ship safety early warning based on dynamic scene change detection, the labor cost and time cost can be reduced, the missed report and false report of ship safety anomalies can be avoided, and the reliability of ship safety monitoring can be improved.

[0077] Optionally, the video segment clustering module 220 is specifically configured to perform frame extraction on each of the video segments to obtain multiple video frame images corresponding to each of the video segments;

[0078] By calculating the similarity degree between multiple video frame images corresponding to adjacent video segments, the mean score and standard deviation score corresponding to each of the video segments are obtained.

[0079] Optionally, the video segment clustering module 220 is specifically configured to obtain the current video frame image corresponding to the current video segment, and obtain the associated video frame image corresponding to the current video frame image among the multiple video frame images corresponding to the video segment adjacent to the current video segment;

[0080] Calculate the similarity degree between the current video frame image and the associated video frame image to obtain the similarity score corresponding to the current video frame image, and obtain the similarity scores corresponding to each video frame image of the current video segment;

[0081] Calculate the mean of the similarity scores corresponding to each video frame image to obtain the mean score corresponding to the current video segment, and calculate the standard deviation of the similarity scores corresponding to each video frame image to obtain the standard deviation score corresponding to the current video segment.

[0082] Optionally, the video segment clustering module 220 is specifically configured to perform feature projection on the current video frame image to obtain a current projection feature map, and perform feature projection on the associated video frame image to obtain an associated projection feature map;

[0083] Calculate the similarity score between the current projection feature map and the associated projection feature map as the similarity score corresponding to the current video frame image.

[0084] Optionally, the video segment clustering module 220 is specifically configured to divide the current projection feature map into cells to obtain a plurality of first cell features, and flatten each of the first cell features to obtain a first feature vector sequence corresponding to each of the first cell features;

[0085] Divide the associated projection feature map into cells to obtain a plurality of second cell features, and flatten each of the second cell features to obtain a second feature vector sequence corresponding to each of the second cell features;

[0086] According to the first feature vector sequence corresponding to each of the first cell features and the second feature vector sequence corresponding to each of the second cell features, if an overlapping feature vector is detected in the current first cell feature, obtain the number of the overlapping feature vectors;

[0087] If the number of the overlapping feature vectors is greater than a preset threshold, determine the current first cell feature as a matching cell feature, and calculate a ratio between the number of the matching cell features and the number of the first cell features as a similarity score between the current projection feature map and the associated projection feature map.

[0088] Optionally, the video segment clustering module 220 is specifically configured to obtain a target second cell feature matched by the current first cell feature according to the position of the current first cell feature in the current projection feature map;

[0089] Obtain a plurality of target first feature vectors according to the occurrence times corresponding to each first feature vector in the first feature vector sequence corresponding to the current first cell feature, and obtain a plurality of target second feature vectors according to the occurrence times corresponding to each second feature vector in the second feature vector sequence corresponding to the target second cell feature;

[0090] If it is detected that a current target first feature vector is equal to a certain target second feature vector and the corresponding occurrence times are equal, determine the current target first feature vector as an overlapping feature vector.

[0091] The ship safety monitoring device provided by the embodiments of the present invention can execute the ship safety monitoring method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0092] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information and other processes all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0093] Embodiment III

[0094] Figure 5 FIG. 1 shows a schematic structural diagram of an electronic device 30 that can be used to implement an embodiment of the present invention. The electronic device 30 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 30 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described herein and / or claimed.

[0095] As Figure 5 shown, the electronic device 30 includes at least one processor 31, and a memory communicatively connected to the at least one processor 31, such as a read-only memory (ROM) 32, a random access memory (RAM) 33, etc. The memory stores a computer program executable by the at least one processor. The processor 31 can perform various appropriate actions and processes according to the computer program stored in the read-only memory 32 or the computer program loaded from the storage unit 38 into the random access memory 33. In the RAM 33, various programs and data required for the operation of the electronic device 30 can also be stored. The processor 31, the ROM 32, and the RAM 33 are connected to each other via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0096] A plurality of components in the electronic device 30 are connected to the I / O interface 35, including: an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a magnetic disk, an optical disk, etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 39 allows the electronic device 30 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0097] The processor 31 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 31 include, but are not limited to, a central processing unit, a graphics processing unit, various dedicated artificial intelligence computing chips, various processors running machine learning model algorithms, a digital signal processor, and any appropriate processor, controller, microcontroller, etc. The processor 31 executes the various methods and processes described above, such as the method for monitoring ship safety.

[0098] In some embodiments, the method for monitoring ship safety can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 30 via the ROM 32 and / or the communication unit 39. When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the method for monitoring ship safety described above can be performed. Alternatively, in other embodiments, the processor 31 can be configured to execute the method for monitoring ship safety by any other suitable means (e.g., by means of firmware).

[0099] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays, application specific integrated circuits, application specific standard products, systems on a chip, complex programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0100] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer programs are executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0101] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0102] In order to provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device 30 having: a display device (e.g., a cathode ray tube or a liquid crystal display) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device 30. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input received from the user can be in any form (including acoustic input, voice input, or tactile input).

[0103] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of the communication network include: a local area network, a wide area network, a blockchain network, and the Internet.

[0104] The computing system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server.

[0105] This embodiment may further include a computer program product, which includes a computer program that, when executed by a processor, implements the ship safety monitoring method provided in any embodiment of the present invention.

[0106] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitations are imposed herein.

[0107] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for monitoring ship safety, characterized in that: include: Acquire a ship monitoring video, and perform window segmentation on the ship monitoring video to obtain multiple video clips; Perform frame extraction processing on each of the video clips to obtain a plurality of video frame images corresponding to each of the video clips; Acquire a current video frame image corresponding to a current video segment, and acquire an associated video frame image corresponding to the current video frame image from a plurality of video frame images corresponding to video segments adjacent to the current video segment; Perform feature projection on the current video frame image to obtain a current projection feature map, and perform feature projection on the associated video frame image to obtain an associated projection feature map; Divide the current projection feature map into cells to obtain a plurality of first cell features, and flatten each of the first cell features to obtain a first feature vector sequence corresponding to each of the first cell features; Divide the associated projection feature map into cells to obtain multiple second cell features, and flatten each of the second cell features to obtain a second feature vector sequence corresponding to each of the second cell features; According to the first feature vector sequence corresponding to each of the first cell features and the second feature vector sequence corresponding to each of the second cell features, if it is detected that there are overlapping feature vectors in the current first cell features, then the number of the overlapping feature vectors is obtained; If the number of overlapping feature vectors is greater than a preset threshold, the current first cell feature is determined as a matching cell feature, and a ratio between the number of matching cell features and the number of the first cell features is calculated as a similarity score between the current projection feature map and the associated projection feature map, a similarity score corresponding to the current video frame image is obtained, and similarity scores corresponding to each video frame image corresponding to the current video clip are obtained; Calculating the mean of the similarity scores corresponding to the video frame images to obtain the mean score corresponding to the current video clip, and calculating the standard deviation of the similarity scores corresponding to the video frame images to obtain the standard deviation score corresponding to the current video clip, and performing cluster analysis on the multiple video clips according to the mean score and the standard deviation score to obtain a video clustering result; According to the video clustering result, if it is detected that the target video segment belongs to the scene change class, a ship safety alarm is issued.

2. The method according to claim 1, characterized in that According to the first feature vector sequence corresponding to each of the first cell features and the second feature vector sequence corresponding to each of the second cell features, detecting that there are overlapping feature vectors in the current first cell features includes: According to the position of the current first cell feature in the current projection feature map, obtain the target second cell feature that matches the current first cell feature; Acquire multiple target first feature vectors according to the number of occurrences of each first feature vector in the first feature vector sequence corresponding to the current first cell feature, and acquire multiple target second feature vectors according to the number of occurrences of each second feature vector in the second feature vector sequence corresponding to the target second cell feature; If it is detected that the current target first feature vector is equal to a certain target second feature vector, and the corresponding number of occurrences is equal, the current target first feature vector is determined as an overlapping feature vector.

3. A ship safety monitoring device, characterized in that: include: A video clip acquisition module is used to acquire a ship monitoring video and divide the ship monitoring video into windows to acquire multiple video clips; A video segment clustering module, used for performing frame extraction processing on each of the video segments to obtain a plurality of video frame images corresponding to each of the video segments; Acquire a current video frame image corresponding to a current video segment, and acquire an associated video frame image corresponding to the current video frame image from a plurality of video frame images corresponding to video segments adjacent to the current video segment; Perform feature projection on the current video frame image to obtain a current projection feature map, and perform feature projection on the associated video frame image to obtain an associated projection feature map; Divide the current projection feature map into cells to obtain a plurality of first cell features, and flatten each of the first cell features to obtain a first feature vector sequence corresponding to each of the first cell features; Divide the associated projection feature map into cells to obtain multiple second cell features, and flatten each of the second cell features to obtain a second feature vector sequence corresponding to each of the second cell features; According to the first feature vector sequence corresponding to each of the first cell features and the second feature vector sequence corresponding to each of the second cell features, if it is detected that there are overlapping feature vectors in the current first cell features, then the number of the overlapping feature vectors is obtained; If the number of overlapping feature vectors is greater than a preset threshold, the current first cell feature is determined as a matching cell feature, and a ratio between the number of matching cell features and the number of the first cell features is calculated as a similarity score between the current projection feature map and the associated projection feature map, a similarity score corresponding to the current video frame image is obtained, and similarity scores corresponding to each video frame image corresponding to the current video clip are obtained; Calculating the mean of the similarity scores corresponding to the video frame images to obtain the mean score corresponding to the current video clip, and calculating the standard deviation of the similarity scores corresponding to the video frame images to obtain the standard deviation score corresponding to the current video clip, and performing cluster analysis on the multiple video clips according to the mean score and the standard deviation score to obtain a video clustering result; The ship safety alarm module is used to issue a ship safety alarm if it is detected that the target video segment belongs to the scene change category according to the video clustering result.

4. An electronic device, characterized in that: The electronic device comprises: at least one processor, and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the ship safety monitoring method according to any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the ship safety monitoring method according to any one of claims 1 to 2 when executed.

6. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the ship safety monitoring method according to any one of claims 1 to 2.

Images