Water surface object monitoring method and device, electronic equipment and storage medium

Abstract

The embodiment of the present application provides a kind of water surface object monitoring method, when detecting suspicious object in water surface monitoring image, obtain historical monitoring image;According to the historical monitoring image and the water surface monitoring image, the first image sequence of first to be measured area is extracted, and the second image sequence of second to be measured area is extracted, the first to be measured area is determined according to the region corresponding inside the suspicious object, and the second to be measured area is determined according to the region corresponding outside the suspicious object;Whether the suspicious object is target object is judged based on the first image sequence and the second image sequence.The image change condition of the internal region of suspicious object and the image change condition of the external region of suspicious object can be obtained by analyzing the first image sequence and the second image sequence of historical monitoring image and water surface monitoring image, to judge whether suspicious object is target object, and the identification accuracy of target object can be extracted.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a water surface object monitoring method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the process of automatically monitoring the water surface by image processing means, due to the generation of the inverted image, it is difficult to distinguish between the inverted image and the entity, especially when the water surface is not calm, some changes in the inverted image will become the shape of the object, making the automatic monitoring of the water surface produce misidentification. For example, due to the change of water surface ripples, part of the water surface area is irregular white, and the automatic monitoring process may identify this part of the area as white garbage. Therefore, the existing automatic monitoring of the water surface has low recognition accuracy. SUMMARY

[0003] The embodiments of the present application provide a water surface object monitoring method, which aims to solve the problem of low recognition accuracy of the existing automatic monitoring of the water surface. By analyzing the historical monitoring image and the first image sequence and the second image sequence of the water surface monitoring image, the image change of the suspicious object internal area and the image change of the suspicious object external area can be obtained, so as to judge whether the suspicious object is the target object, and the recognition accuracy of the target object can be extracted.

[0004] In a first aspect, the embodiments of the present application provide a water surface object monitoring method, which comprises:

[0005] When a suspicious object is detected in the water surface monitoring image, a historical monitoring image is acquired;

[0006] According to the historical monitoring image and the water surface monitoring image, a first image sequence of a first to-be-tested area and a second image sequence of a second to-be-tested area are extracted, the first to-be-tested area is determined according to the corresponding area inside the suspicious object, and the second to-be-tested area is determined according to the corresponding area outside the suspicious object;

[0007] Based on the first image sequence and the second image sequence, it is judged whether the suspicious object is a target object.

[0008] Optionally, before the first image sequence of the first to-be-tested area and the second image sequence of the second to-be-tested area are extracted according to the historical monitoring image and the water surface monitoring image, the method further comprises:

[0009] The water surface monitoring image is subjected to image segmentation to obtain a segmentation contour of the suspicious object;

[0010] According to the segmentation contour of the suspicious object, a corresponding region inside the suspicious object is determined in the water surface monitoring image, and a corresponding region outside the suspicious object is determined in the water surface monitoring image.

[0011] Optionally, the extracting a first image sequence of a first to-be-tested region and a second image sequence of a second to-be-tested region according to the historical monitoring image and the water surface monitoring image comprises:

[0012] According to the corresponding region inside the suspicious object, a first region in the water surface monitoring image is determined, and according to the corresponding region outside the suspicious object, a second region in the water surface monitoring image is determined;

[0013] The corresponding region inside the suspicious object is mapped to the historical monitoring image to obtain a first region in the historical monitoring image, and the corresponding region outside the suspicious object is mapped to the historical monitoring image to obtain a second region in the historical monitoring image;

[0014] According to the first region in the water surface monitoring image and the first region in the historical monitoring image, a first to-be-tested region is determined, and according to the second region in the water surface monitoring image and the second region in the historical monitoring image, a second to-be-tested region is determined;

[0015] Image extraction is performed on the first to-be-tested region in the historical monitoring image and the water surface monitoring image to obtain a first image sequence, and image extraction is performed on the second to-be-tested region in the historical monitoring image and the water surface monitoring image to obtain a second image sequence.

[0016] Optionally, the judging whether the suspicious object is a target object based on the first image sequence and the second image sequence comprises:

[0017] Temporal and spatial feature extraction is performed on the first image sequence to obtain temporal and spatial features of the first image sequence, and temporal and spatial feature extraction is performed on the second image sequence to obtain temporal and spatial features of the second image sequence;

[0018] The judging whether the suspicious object is a target object based on the temporal and spatial features of the first image sequence and the temporal and spatial features of the second image sequence.

[0019] Optionally, the temporal and spatial feature extraction on the first image sequence to obtain the temporal and spatial features of the first image sequence and the temporal and spatial feature extraction on the second image sequence to obtain the temporal and spatial features of the second image sequence comprise:

[0020] extracting spatial features of each frame image in the first image sequence to obtain spatial features of each frame image in the first image sequence;

[0021] splicing the spatial features of each frame image in the first image sequence in the order of corresponding frame images in the first image sequence to obtain spatio-temporal features of the first image sequence;

[0022] extracting spatial features of each frame image in the second image sequence to obtain second spatial features of each frame image in the second image sequence;

[0023] splicing the spatial features of each frame image in the second image sequence in the order of corresponding frame images in the second image sequence to obtain spatio-temporal features of the second image sequence.

[0024] Optionally, the judging whether the suspicious object is a target object based on the spatio-temporal features of the first image sequence and the spatio-temporal features of the second image sequence comprises:

[0025] transforming the spatio-temporal features of the first image sequence and the spatio-temporal features of the second image sequence into frequency domains respectively to obtain frequency domain features of the first image sequence and frequency domain features of the second image sequence;

[0026] judging whether the suspicious object is a target object based on the frequency domain features of the first image sequence and the frequency domain features of the second image sequence.

[0027] Optionally, the judging whether the suspicious object is a target object based on the frequency domain features of the first image sequence and the frequency domain features of the second image sequence comprises:

[0028] calculating a similarity between the frequency domain features of the first image sequence and the frequency domain features of the second image sequence;

[0029] if the similarity is less than a preset value, determining that the suspicious object is a target object;

[0030] if the similarity is greater than the preset value, determining that the suspicious object is a non-target object.

[0031] In a second aspect, an embodiment of the present application provides a water surface object monitoring device, which comprises:

[0032] an acquisition module configured to acquire historical monitoring images when a suspicious object is detected in a water surface monitoring image;

[0033] extract a first image sequence of a first to-be-measured region according to the historical monitoring image and the water surface monitoring image, the first to-be-measured region being determined according to a corresponding region inside the suspicious object, and extract a second image sequence of a second to-be-measured region according to the historical monitoring image and the water surface monitoring image, the second to-be-measured region being determined according to a corresponding region outside the suspicious object;

[0034] a judging module configured to judge whether the suspicious object is a target object based on the first image sequence and the second image sequence.

[0035] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps in the water surface object monitoring method provided by the embodiments of the present application when executing the computer program.

[0036] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps in the water surface object monitoring method provided by the embodiments of the present application when executed by a processor.

[0037] In the embodiments of the present application, when a suspicious object is detected in a water surface monitoring image, a historical monitoring image is acquired; a first image sequence of a first to-be-measured region is extracted according to the historical monitoring image and the water surface monitoring image, the first to-be-measured region being determined according to a corresponding region inside the suspicious object, and a second image sequence of a second to-be-measured region is extracted according to the historical monitoring image and the water surface monitoring image, the second to-be-measured region being determined according to a corresponding region outside the suspicious object; and whether the suspicious object is a target object is judged based on the first image sequence and the second image sequence. By analyzing the first image sequence and the second image sequence of the historical monitoring image and the water surface monitoring image, the image change of the region inside the suspicious object and the image change of the region outside the suspicious object can be obtained, so as to judge whether the suspicious object is a target object, and the recognition accuracy of the target object can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0039] Figure 1 is a flowchart of a water surface object monitoring method provided by the embodiments of the present application;

[0040] Figure 2is a structural schematic diagram of a water surface object monitoring device provided by an embodiment of the present application.

[0041] Figure 3 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0043] Please refer to Figure 1 , Figure 1 is a flowchart of a water surface object monitoring method provided by an embodiment of the present application, as shown in the figure, the water surface object monitoring method comprises the following steps: Figure 1

[0044] 101, when a suspicious object is detected in a water surface monitoring image, a historical monitoring image is acquired.

[0045] In the embodiments of the present application, the above-mentioned water surface object monitoring method can be used for object monitoring of a water channel such as a river surface or a lake surface, and can be specifically used for garbage detection or biological detection of the water channel such as the river surface or the lake surface. The above-mentioned water surface monitoring image can be obtained by photographing a target water surface area through a camera fixedly arranged on a shore.

[0046] The above-mentioned water surface monitoring image can be understood as a water surface monitoring image at a current time, and the above-mentioned historical monitoring image can be understood as an image before the current time.

[0047] The water surface monitoring image can be detected in real time through a target detection algorithm, so as to determine whether there is a suspicious object in the water surface monitoring image. When there is a suspicious object in the water surface monitoring image, it can be indicated that a suspicious object is detected in the water surface monitoring image. The above-mentioned target detection algorithm can be an existing target detection algorithm, which can be a target detection algorithm based on YOLOV series.

[0048] The above-mentioned suspicious object can be a garbage type object or a biological type object. The garbage type object can be garbage such as a plastic bag, a fast food box, a wooden board, etc. The biological type can be water birds, fish and other biological objects. The monitoring image can be stored in a storage or a server. When a suspicious object is detected in the water surface monitoring image, the historical monitoring image can be searched in the storage or the server.

[0049] ​102、extracting a first image sequence of the first to-be-measured region and a second image sequence of the second to-be-measured region according to the historical monitoring image and the water surface monitoring image.

[0050] In the embodiment of the present application, the first to-be-measured region is determined according to the region corresponding to the inside of the suspicious object, and the second to-be-measured region is determined according to the region corresponding to the outside of the suspicious object. The inside of the suspicious object is the region within the suspicious object contour in the water surface monitoring image, and the outside of the suspicious object is the region outside the suspicious object contour in the water surface monitoring image.

[0051] The position of the first to-be-measured region in the water surface monitoring image is the same as that in the historical monitoring image, and the position of the second to-be-measured region in the water surface monitoring image is the same as that in the historical monitoring image.

[0052] Specifically, the first image sequence includes the image region of the suspicious object in the water surface monitoring image and the image region at the same position as the suspicious object in the historical monitoring image, and the second image sequence includes the image region around the suspicious object in the water surface monitoring image and the image region at the same position as the suspicious object in the historical monitoring image.

[0053] 103、judging whether the suspicious object is the target object based on the first image sequence and the second image sequence.

[0054] In the embodiment of the present application, the first image sequence records the image change information of the region where the inside of the suspicious object is located, and the second image sequence records the image change information of the region where the outside of the suspicious object is located. Therefore, the image change information of the region where the inside of the suspicious object is located can be compared with the image change information of the region where the outside of the suspicious object is located to judge whether the suspicious object is the target object.

[0055] It should be noted that if the suspicious object is caused by water surface fluctuation, the region corresponding to the inside of the suspicious object and the region corresponding to the outside of the suspicious object will have similar change trends, and if the suspicious object is the target object, the region corresponding to the inside of the suspicious object and the region corresponding to the outside of the suspicious object will have different change trends.

[0056] In the embodiment of the present application, when a suspicious object is detected in a water surface monitoring image, a historical monitoring image is acquired; a first image sequence of a first to-be-detected region is extracted and a second image sequence of a second to-be-detected region is extracted according to the historical monitoring image and the water surface monitoring image, the first to-be-detected region is determined according to a region corresponding to the inside of the suspicious object, and the second to-be-detected region is determined according to a region corresponding to the outside of the suspicious object; whether the suspicious object is a target object is judged based on the first image sequence and the second image sequence. By analyzing the historical monitoring image and the first image sequence and the second image sequence of the water surface monitoring image, the image change of the region inside the suspicious object and the image change of the region outside the suspicious object can be obtained, so as to judge whether the suspicious object is the target object, and the recognition accuracy of the target object can be improved.

[0057] Optionally, before the step of extracting the first image sequence of the first to-be-detected region and the second image sequence of the second to-be-detected region according to the historical monitoring image and the water surface monitoring image, the water surface monitoring image can be subjected to image segmentation to obtain a segmentation contour of the suspicious object; the region corresponding to the inside of the suspicious object in the water surface monitoring image is determined according to the segmentation contour of the suspicious object, and the region corresponding to the outside of the suspicious object in the water surface monitoring image is determined.

[0058] In the embodiment of the present application, the suspicious object can be subjected to image segmentation by using an existing image segmentation algorithm to obtain a segmentation contour of the suspicious object. The inside of the segmentation contour of the suspicious object is the inside of the suspicious object, and the outside of the segmentation contour of the suspicious object is the outside of the suspicious object. Then, according to the position of the segmentation contour of the suspicious object in the water surface monitoring image, the region corresponding to the inside of the suspicious object in the water surface monitoring image is determined, and the region corresponding to the outside of the suspicious object in the water surface monitoring image is determined. The region corresponding to the inside of the suspicious object and the region corresponding to the outside of the suspicious object are divided by the segmentation contour line of the suspicious object.

[0059] Further, the region corresponding to the outside of the suspicious object can be obtained by expanding the segmentation contour of the suspicious object, and the region corresponding to the outside of the suspicious object can be expanded outward based on the segmentation contour of the suspicious object. The area of the outward expansion can be the area of the suspicious object in the water surface monitoring image.

[0060] Optionally, in the step of extracting the first image sequence of the first to-be-measured region and the second image sequence of the second to-be-measured region according to the historical monitoring image and the water surface monitoring image, the first region in the water surface monitoring image can be determined according to the region corresponding to the inside of the suspicious object, and the first region in the water surface monitoring image can be determined according to the region corresponding to the outside of the suspicious object; the region corresponding to the inside of the suspicious object is mapped to the historical monitoring image to obtain the first region in the historical monitoring image, and the region corresponding to the outside of the suspicious object is mapped to the historical monitoring image to obtain the second region in the historical monitoring image; the first to-be-measured region is determined according to the first region in the water surface monitoring image and the first region in the historical monitoring image, and the second to-be-measured region is determined according to the second region in the water surface monitoring image and the second region in the historical monitoring image; the first image sequence is obtained by image extraction in the first to-be-measured region in the historical monitoring image and the water surface monitoring image, and the second image sequence of the second to-be-measured region is extracted in the historical monitoring and the water surface monitoring image.

[0061] In the embodiment of the present application, the region corresponding to the inside of the suspicious object in the water surface monitoring image can be determined through the segmentation contour of the suspicious object, and the region is determined as the first region in the water surface monitoring image. The region corresponding to the outside of the suspicious object in the water surface monitoring image can be determined through the segmentation contour of the suspicious object, and the region is determined as the second region in the water surface monitoring image. The first region in the water surface monitoring image and the second region in the water surface monitoring image are divided by the segmentation contour of the suspicious object.

[0062] It should be noted that the water surface monitoring image and the historical monitoring image have the same image resolution size, and the water surface monitoring image and the historical monitoring image are images captured by the same camera, so the water surface monitoring image and the historical monitoring image have the same image coordinate system. Therefore, the region corresponding to the inside of the suspicious object can be mapped to the historical monitoring image to obtain the first region in the historical monitoring image, and the region corresponding to the outside of the suspicious object can be mapped to the historical monitoring image to obtain the second region in the historical monitoring image. In this way, the first region in the water surface monitoring image and the first region in the historical monitoring image have the same image position, and the second region in the water surface monitoring image and the second region in the historical monitoring image have the same image position.

[0063] The first region in the water surface monitoring image and the first region in the historical monitoring image are combined in time sequence to obtain a first to-be-tested region, and the second region in the water surface monitoring image and the second region in the historical monitoring image are combined in time sequence to obtain a second to-be-tested region. The first to-be-tested region is subjected to image extraction to obtain a first image sequence, and the second to-be-tested region is subjected to image extraction to obtain a second image sequence. The first image sequence includes image change information of a corresponding region inside the suspicious object, and the second image sequence includes image change information of a corresponding region outside the suspicious object.

[0064] Optionally, in the step of judging whether the suspicious object is the target object based on the first image sequence and the second image sequence, spatio-temporal feature extraction can be performed on the first image sequence to obtain spatio-temporal features of the first image sequence, and spatio-temporal feature extraction can be performed on the second image sequence to obtain spatio-temporal features of the second image sequence; whether the suspicious object is the target object is judged based on the spatio-temporal features of the first image sequence and the spatio-temporal features of the second image sequence.

[0065] In the embodiment of the application, the spatio-temporal feature extraction can be performed on the first image sequence and the second image sequence by the feature extraction engine. The spatio-temporal features include time sequence features and spatial features. The time sequence features can be the time dimension in the image sequence, and the spatial features can be the spatial distribution of the pixel points of each frame of image in the image sequence. Since the image sequence includes image change information, the image change information can be abstracted by extracting the spatio-temporal features of the image sequence, and the image change information can be expressed by a higher-level feature language.

[0066] The feature extraction engine can be a feature extraction part in an image recognition model, such as a feature extraction network in the YOLOV series.

[0067] It should be noted that the spatio-temporal features of the first image sequence express the change of the pixel point distribution in the internal region of the suspicious object, and the spatio-temporal features of the second image sequence express the change of the pixel point distribution in the external region of the suspicious object. If the suspicious object is caused by water surface ripples, the change of the pixel point distribution in the internal region of the suspicious object is similar to the change of the pixel point distribution in the external region of the suspicious object. If the suspicious object is not caused by water surface ripples, there is a certain difference between the change of the pixel point distribution in the internal region of the suspicious object and the change of the pixel point distribution in the external region of the suspicious object. Therefore, whether the suspicious object is the target object can be judged by calculating the similarity of the spatio-temporal features of the first image sequence and the spatio-temporal features of the second image sequence.

[0068] Optionally, in the steps of performing spatio-temporal feature extraction on the first image sequence to obtain the spatio-temporal features of the first image sequence and performing spatio-temporal feature extraction on the second image sequence to obtain the spatio-temporal features of the second image sequence, the spatial feature of each frame of image in the first image sequence can be extracted to obtain the spatial feature of each frame of image in the first image sequence; the spatial features of each frame of image in the first image sequence are spliced in the order of the corresponding frame of image in the first image sequence to obtain the spatio-temporal features of the first image sequence; the spatial feature of each frame of image in the second image sequence is extracted to obtain the spatial feature of each frame of image in the second image sequence; the spatial features of each frame of image in the second image sequence are spliced in the order of the corresponding frame of image in the second image sequence to obtain the spatio-temporal features of the second image sequence.

[0069] In the embodiment of the application, the spatial feature of each frame of image in the image sequence can be extracted by the feature extraction engine to obtain the spatial feature of each frame of image in the image sequence, and the spatial feature of a frame of image corresponds to the spatial distribution of the pixel points in the frame of image. The spatial feature of each frame of image can be understood as a feature vector, and the spatial features of each frame of image are vector spliced in the order of the frames of image in the image sequence, so that the spatio-temporal features of the image sequence can be obtained.

[0070] It should be noted that the first image sequence and the second image sequence have the same time dimension, which can be understood as that each frame of image in the first image sequence has a corresponding frame of image in the second image sequence.

[0071] Optionally, in the step of judging whether the suspicious object is the target object based on the spatio-temporal features of the first image sequence and the spatio-temporal features of the second image sequence, the spatio-temporal features of the first image sequence and the spatio-temporal features of the second image sequence can be transformed into the frequency domain to obtain the frequency domain features of the first image sequence and the frequency domain features of the second image sequence; and whether the suspicious object is the target object is judged based on the frequency domain features of the first image sequence and the frequency domain features of the second image sequence.

[0072] In the embodiment of the present application, it is considered that the water surface ripples cause false recognition for the automatic monitoring of the water surface, and the influence of the water surface ripples on the image is an influence of ripple change, so that the image change information is subject to this ripple change. Since the target object has its own structural stability, this ripple change only affects the water surface, but does not affect the target object, and if the spatio-temporal features of the first image sequence are subject to this ripple change, it can be judged that the suspicious object is a false recognition caused by the water surface ripples, and if the spatio-temporal features of the first image sequence are not subject to this ripple change, it can be judged that the suspicious object is a target object. Further, since the second image sequence is an image sequence corresponding to the external region of the suspicious object, the second image sequence can be understood as an image sequence of the water surface, and therefore, this ripple change can be reflected in the spatio-temporal features of the second image sequence.

[0073] The spatio-temporal features of the first image sequence can be converted from the time domain to the frequency domain by Fourier transform to obtain the frequency domain features of the first image sequence, and the spatio-temporal features of the second image sequence can be converted from the time domain to the frequency domain by Fourier transform to obtain the frequency domain features of the second image sequence. The water surface ripples can be converted into a waveform distribution of different frequencies and amplitudes, i.e. frequency domain features, to describe the water surface ripple change rule through this waveform distribution.

[0074] Since the frequency domain features of the second image sequence represent the water surface waveform change, if the frequency domain features of the first image sequence are subject to the water surface waveform change corresponding to the second image sequence, it means that the suspicious object is a false recognition caused by the water surface ripples, and if the frequency domain features of the first image sequence are not subject to the water surface waveform change corresponding to the second image sequence, it means that the suspicious object is a target object.

[0075] Optionally, in the step of judging whether the suspicious object is a target object based on the frequency domain features of the first image sequence and the frequency domain features of the second image sequence, the similarity between the frequency domain features of the first image sequence and the frequency domain features of the second image sequence can be calculated; if the similarity is less than a preset value, it is determined that the suspicious object is a target object; and if the similarity is greater than the preset value, it is determined that the suspicious object is a non-target object.

[0076] In the embodiment of the present application, after the frequency domain features of the first image sequence and the frequency domain features of the second image sequence are obtained, the cosine similarity between the frequency domain features of the first image sequence and the frequency domain features of the second image sequence can be calculated by using the cosine similarity algorithm. If the cosine similarity is less than a preset value, it indicates that the frequency domain features of the first image sequence and the frequency domain features of the second image sequence are not similar, that is, the frequency domain features of the first image sequence do not conform to the water surface wave shape change corresponding to the second image sequence. Therefore, it can be determined that the suspicious object is a target object. If the cosine similarity is greater than the preset value, it indicates that the frequency domain features of the first image sequence and the frequency domain features of the second image sequence are relatively similar, that is, the frequency domain features of the first image sequence conform to the water surface wave shape change corresponding to the second image sequence. Therefore, it can be determined that the suspicious object is a non-target object, that is, the suspicious object is a misrecognition caused by water surface ripples.

[0077] The preset value described above can be set according to the wind power of the monitoring environment. Specifically, if the monitoring environment is a windy environment, water surface ripples are generated more frequently. Due to the generation of more water surface ripples, the data amount of water surface ripples in the image sequence is also larger, so that the internal law of water surface wave shape change is more stable. Therefore, the preset value can be set to be larger, for example, set to 0.98. If the monitoring environment is a less windy environment, water surface ripples are not generated frequently. Due to the generation of less water surface ripples, the data amount of water surface ripples in the image sequence is also smaller, so that the internal law of water surface wave shape change is less stable. Therefore, the preset value can be set to be smaller, for example, set to 0.92.

[0078] It should be noted that the water surface object monitoring method provided by the embodiment of the present application can be applied to intelligent cameras, smart phones, computers, servers and other devices that can perform water surface object monitoring methods.

[0079] Optionally, please refer to Figure 2 , Figure 2 is a structural schematic diagram of a water surface object monitoring device provided by the embodiment of the present application, as Figure 2 shown, the device comprises:

[0080] The acquisition module 201 is configured to acquire a historical monitoring image when a suspicious object is detected in a water surface monitoring image.

[0081] The extraction module 202 is configured to extract a first image sequence of a first to-be-measured region and a second image sequence of a second to-be-measured region according to the historical monitoring image and the water surface monitoring image, the first to-be-measured region being determined according to a region corresponding to the inside of the suspicious object, and the second to-be-measured region being determined according to a region corresponding to the outside of the suspicious object.

[0082] The judging module 203 is configured to judge whether the suspicious object is a target object based on the first image sequence and the second image sequence.

[0083] Optionally, the device further comprises:

[0084] The dividing module is configured to perform image division on the water surface monitoring image to obtain a division contour of the suspicious object.

[0085] The determining module is configured to determine a corresponding region inside the suspicious object in the water surface monitoring image and a corresponding region outside the suspicious object in the water surface monitoring image according to the division contour of the suspicious object.

[0086] Optionally, the extracting module 202 comprises:

[0087] The first determining sub-module is configured to determine a first region in the water surface monitoring image according to the corresponding region inside the suspicious object and determine a second region in the water surface monitoring image according to the corresponding region outside the suspicious object.

[0088] The mapping sub-module is configured to map the corresponding region inside the suspicious object to the historical monitoring image to obtain a first region in the historical monitoring image and map the corresponding region outside the suspicious object to the historical monitoring image to obtain a second region in the historical monitoring image.

[0089] The second determining sub-module is configured to determine a first to-be-tested region according to the first region in the water surface monitoring image and the first region in the historical monitoring image and determine a second to-be-tested region according to the second region in the water surface monitoring image and the second region in the historical monitoring image.

[0090] The first extracting sub-module is configured to perform image extraction on the first to-be-tested region in the historical monitoring image and the water surface monitoring image to obtain a first image sequence and perform image extraction on the second to-be-tested region in the historical monitoring image and the water surface monitoring image to obtain a second image sequence.

[0091] Optionally, the judging module 203 comprises:

[0092] The second extracting sub-module is configured to perform spatio-temporal feature extraction on the first image sequence to obtain spatio-temporal features of the first image sequence and perform spatio-temporal feature extraction on the second image sequence to obtain spatio-temporal features of the second image sequence.

[0093] The judging sub-module is configured to judge whether the suspicious object is a target object based on the spatio-temporal features of the first image sequence and the spatio-temporal features of the second image sequence.

[0094] Optionally, the second extraction submodule comprises:

[0095] a first extraction unit, configured to perform spatial feature extraction on each frame of image in the first image sequence to obtain spatial features of each frame of image in the first image sequence;

[0096] a first splicing unit, configured to splice the spatial features of each frame of image in the first image sequence according to the order of the corresponding frame of image in the first image sequence to obtain the spatiotemporal features of the first image sequence;

[0097] a second extraction unit, configured to perform spatial feature extraction on each frame of image in the second image sequence to obtain second spatial features of each frame of image in the second image sequence;

[0098] a second splicing unit, configured to splice the spatial features of each frame of image in the second image sequence according to the order of the corresponding frame of image in the second image sequence to obtain the spatiotemporal features of the second image sequence.

[0099] Optionally, the judging submodule comprises:

[0100] a transformation unit, configured to transform the spatiotemporal features of the first image sequence and the spatiotemporal features of the second image sequence into the frequency domain respectively to obtain the frequency domain features of the first image sequence and the frequency domain features of the second image sequence;

[0101] a judging unit, configured to judge whether the suspicious object is a target object based on the frequency domain features of the first image sequence and the frequency domain features of the second image sequence.

[0102] Optionally, the judging unit comprises:

[0103] a calculation subunit, configured to calculate the similarity between the frequency domain features of the first image sequence and the frequency domain features of the second image sequence;

[0104] a first determination subunit, configured to determine that the suspicious object is a target object if the similarity is less than a preset value;

[0105] a second determination subunit, configured to determine that the suspicious object is a non-target object if the similarity is greater than a preset value.

[0106] It should be noted that the water surface object monitoring device provided by the embodiments of the present application can be applied to intelligent cameras, smart phones, computers, servers and other devices that can perform the water surface object monitoring method.

[0107] The water surface object monitoring device provided by the embodiments of the present application can realize each process of the water surface object monitoring method in the method embodiments, and can achieve the same beneficial effects. To avoid repetition, details are not described herein.

[0108] Referring to Figure 3 , Figure 3 is a structural schematic diagram of an electronic device provided by the embodiments of the present application, as shown in Figure 3 , comprising a memory 302, a processor 301, and a computer program of a water surface object monitoring method stored in the memory 302 and executable on the processor 301, wherein:

[0109] The processor 301 is configured to invoke the computer program stored in the memory 302 to perform the following steps:

[0110] When a suspicious object is detected in a water surface monitoring image, a historical monitoring image is acquired;

[0111] According to the historical monitoring image and the water surface monitoring image, a first image sequence of a first to-be-detected region is extracted, and a second image sequence of a second to-be-detected region is extracted, the first to-be-detected region being determined according to a region corresponding to the inside of the suspicious object, and the second to-be-detected region being determined according to a region corresponding to the outside of the suspicious object;

[0112] Based on the first image sequence and the second image sequence, it is determined whether the suspicious object is a target object.

[0113] Optionally, before the extraction of the first image sequence of the first to-be-detected region and the second image sequence of the second to-be-detected region according to the historical monitoring image and the water surface monitoring image, the method executed by the processor 301 further comprises:

[0114] The water surface monitoring image is subjected to image segmentation to obtain a segmentation contour of the suspicious object;

[0115] According to the segmentation contour of the suspicious object, a region corresponding to the inside of the suspicious object in the water surface monitoring image is determined, and a region corresponding to the outside of the suspicious object in the water surface monitoring image is determined.

[0116] Optionally, the extraction of the first image sequence of the first to-be-detected region and the second image sequence of the second to-be-detected region according to the historical monitoring image and the water surface monitoring image by the processor 301 comprises:

[0117] The first region in the water surface monitoring image is determined according to the region corresponding to the inside of the suspicious object, and the second region in the water surface monitoring image is determined according to the region corresponding to the outside of the suspicious object;

[0118] mapping a region corresponding to the inside of the suspicious object to the historical monitoring image to obtain a first region in the historical monitoring image, and mapping a region corresponding to the outside of the suspicious object to the historical monitoring image to obtain a second region in the historical monitoring image;

[0119] determining a first to-be-detected region according to the first region in the water surface monitoring image and the first region in the historical monitoring image, and determining a second to-be-detected region according to the second region in the water surface monitoring image and the second region in the historical monitoring image;

[0120] extracting a first image sequence from the first to-be-detected region in the historical monitoring image and the water surface monitoring image, and extracting a second image sequence from the second to-be-detected region in the historical monitoring image and the water surface monitoring image.

[0121] Optionally, the determining whether the suspicious object is a target object based on the first image sequence and the second image sequence, performed by the processor 301, includes:

[0122] extracting a spatio-temporal feature of the first image sequence to obtain a spatio-temporal feature of the first image sequence, and extracting a spatio-temporal feature of the second image sequence to obtain a spatio-temporal feature of the second image sequence;

[0123] determining whether the suspicious object is a target object based on the spatio-temporal feature of the first image sequence and the spatio-temporal feature of the second image sequence.

[0124] Optionally, the extracting a spatio-temporal feature of the first image sequence to obtain a spatio-temporal feature of the first image sequence, and extracting a spatio-temporal feature of the second image sequence to obtain a spatio-temporal feature of the second image sequence, performed by the processor 301, includes:

[0125] extracting a spatial feature of each frame of image in the first image sequence to obtain a spatial feature of each frame of image in the first image sequence;

[0126] splicing the spatial feature of each frame of image in the first image sequence according to the order of the corresponding frame of image in the first image sequence to obtain the spatio-temporal feature of the first image sequence;

[0127] extracting a spatial feature of each frame of image in the second image sequence to obtain a second spatial feature of each frame of image in the second image sequence;

[0128] The spatial features of each frame image in the second image sequence are spliced according to the order of the corresponding frame image in the second image sequence, to obtain the space-time features of the second image sequence.

[0129] Optionally, the processor 301 performs the judging whether the suspicious object is the target object based on the space-time features of the first image sequence and the space-time features of the second image sequence, including:

[0130] The space-time features of the first image sequence and the space-time features of the second image sequence are transformed into the frequency domain, to obtain the frequency domain features of the first image sequence and the frequency domain features of the second image sequence.

[0131] The judging whether the suspicious object is the target object is based on the frequency domain features of the first image sequence and the frequency domain features of the second image sequence.

[0132] Optionally, the processor 301 performs the judging whether the suspicious object is the target object based on the frequency domain features of the first image sequence and the frequency domain features of the second image sequence, including:

[0133] The similarity between the frequency domain features of the first image sequence and the frequency domain features of the second image sequence is calculated.

[0134] If the similarity is less than a preset value, it is determined that the suspicious object is the target object.

[0135] If the similarity is greater than the preset value, it is determined that the suspicious object is a non-target object.

[0136] The electronic device provided by the embodiment of the present application can realize each process of the water surface object monitoring method in the above-mentioned method embodiment, and can achieve the same beneficial effects. To avoid repetition, details are not repeated here.

[0137] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to realize each process of the water surface object monitoring method provided by the embodiment of the present application, and can achieve the same technical effects. To avoid repetition, details are not repeated here.

[0138] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware. The program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM).

[0139] The above disclosure is only the preferred embodiments of the present application, and of course cannot limit the scope of the present application. Any equivalent changes made according to the claims of the present application are still within the scope of the present application.

Claims

1. A method for monitoring objects on the water surface, characterized in that, Includes the following steps: When a suspicious object is detected in the water surface monitoring image, historical monitoring images are acquired; The water surface monitoring image is segmented to obtain the segmentation outline of the suspicious object; based on the segmentation outline of the suspicious object, the region corresponding to the interior of the suspicious object and the region corresponding to the exterior of the suspicious object are determined in the water surface monitoring image. A first region in the water surface monitoring image is determined based on the region corresponding to the interior of the suspicious object, and a second region in the water surface monitoring image is determined based on the region corresponding to the exterior of the suspicious object. The region corresponding to the interior of the suspicious object is mapped onto the historical surveillance image to obtain a first region in the historical surveillance image, and the region corresponding to the exterior of the suspicious object is mapped onto the historical surveillance image to obtain a second region in the historical surveillance image. A first area to be tested is determined based on a first area in the water surface monitoring image and a first area in the historical monitoring image; and a second area to be tested is determined based on a second area in the water surface monitoring image and a second area in the historical monitoring image. Image extraction is performed on the first region to be tested in the historical monitoring image and the water surface monitoring image to obtain a first image sequence, and a second image sequence is extracted from the second region to be tested in the historical monitoring image and the water surface monitoring image; the segmentation contour of the suspicious object is obtained by image segmentation algorithm to segment the suspicious object, the inside of the segmentation contour of the suspicious object is the inside of the suspicious object, and the outside of the segmentation contour of the suspicious object is the outside of the suspicious object. Based on the first image sequence and the second image sequence, it is determined whether the suspicious object is the target object. Specifically, spatial features are extracted from each frame of the first image sequence to obtain the spatial features of each frame of the first image sequence. The spatial features of each frame of the first image sequence are then spliced ​​together according to the order of the corresponding frame images in the first image sequence to obtain the spatiotemporal features of the first image sequence. Spatial features are extracted from each frame of the second image sequence to obtain the second spatial features of each frame of the second image sequence; The spatial features of each frame in the second image sequence are spliced ​​together according to the order of the corresponding frame images in the second image sequence to obtain the spatiotemporal features of the second image sequence; the spatiotemporal features of the first image sequence and the second image sequence are transformed into the frequency domain to obtain the frequency domain features of the first image sequence and the frequency domain features of the second image sequence; based on the frequency domain features of the first image sequence and the frequency domain features of the second image sequence, it is determined whether the suspicious object is the target object.

2. The method for monitoring objects on the water surface as described in claim 1, characterized in that, The step of determining whether the suspicious object is the target object based on the frequency domain features of the first image sequence and the frequency domain features of the second image sequence includes: Calculate the similarity between the frequency domain features of the first image sequence and the frequency domain features of the second image sequence; If the similarity is less than a preset value, then the suspicious object is determined to be the target object; If the similarity is greater than a preset value, then the suspicious object is determined to be a non-target object.

3. A device for monitoring objects on the water surface, characterized in that, The device includes: The acquisition module is used to acquire historical monitoring images when a suspicious object is detected in the water surface monitoring images; An extraction module is used to perform image segmentation on the water surface monitoring image to obtain the segmented contour of the suspicious object; based on the segmented contour of the suspicious object, to determine the region corresponding to the interior of the suspicious object and the region corresponding to the exterior of the suspicious object in the water surface monitoring image; to determine a first region in the water surface monitoring image based on the region corresponding to the interior of the suspicious object and a second region in the water surface monitoring image based on the region corresponding to the exterior of the suspicious object; to map the region corresponding to the interior of the suspicious object onto the historical monitoring image to obtain the first region in the historical monitoring image, and to map the region corresponding to the exterior of the suspicious object onto the historical monitoring image to obtain the historical monitoring image. The second region in the image; based on the first region in the water surface monitoring image and the first region in the historical monitoring image, a first region to be tested is determined, and based on the second region in the water surface monitoring image and the second region in the historical monitoring image, a second region to be tested is determined; image extraction is performed on the first region to be tested in the historical monitoring image and the water surface monitoring image to obtain a first image sequence, and a second image sequence is extracted from the historical monitoring image and the water surface monitoring image to obtain the second region to be tested; the segmentation contour of the suspicious object is obtained by image segmentation algorithm to segment the suspicious object, the interior of the segmentation contour of the suspicious object is the interior of the suspicious object, and the exterior of the segmentation contour of the suspicious object is the exterior of the suspicious object; The judgment module is specifically used to: extract spatial features from each frame of the first image sequence to obtain the spatial features of each frame of the first image sequence; concatenate the spatial features of each frame of the first image sequence according to the order of the corresponding frame images in the first image sequence to obtain the spatiotemporal features of the first image sequence; extract spatial features from each frame of the second image sequence to obtain the second spatial features of each frame of the second image sequence; concatenate the spatial features of each frame of the second image sequence according to the order of the corresponding frame images in the second image sequence to obtain the spatiotemporal features of the second image sequence; transform the spatiotemporal features of the first image sequence and the spatiotemporal features of the second image sequence into the frequency domain to obtain the frequency domain features of the first image sequence and the frequency domain features of the second image sequence; and determine whether the suspicious object is the target object based on the frequency domain features of the first image sequence and the frequency domain features of the second image sequence.

4. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the water surface object monitoring method as described in any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the water surface object monitoring method as described in any one of claims 1 to 2.

Images