Intelligent compression method of surveillance video based on ship satellite network edge computing

Abstract

The present application discloses a monitoring video intelligent compression method based on edge computing of a ship satellite network, and relates to the field of signal coding technology. The method comprises: acquiring a monitoring video in a ship satellite network application environment; dividing the monitoring video to obtain at least two data segments; acquiring a first data segment based on at least two data segments; acquiring a first time-series image and a second time-series image based on the first data segment; acquiring a second feature similarity between the first time-series image and the second time-series image; acquiring a division degree based on the second feature similarity and the network congestion degree; encoding the first data segment based on the division degree. The present application analyzes the changes in adjacent time-series images of the monitoring video, performs adaptive segmentation, establishes a segmented three-dimensional model, and analyzes the changes in the feature points of the segment, thereby improving the compression efficiency of video encoding while reducing the computational complexity of edge computing.

Description

Technical Field

[0001] The present application relates to the field of signal coding technology, and specifically to a monitoring video intelligent compression method based on ship satellite network edge computing. Background Art

[0002] During the navigation process of ships, ship satellite networks are often used, which have higher communication delays and lower data transmission rates than ordinary networks. In order to ensure the safety of navigation, various areas of the ship will be monitored, and the existing video surveillance technology based on cloud computing models has high requirements for network bandwidth. At the same time, the burden on the video server of the cloud computing center is increasing, so the requirements for ship satellite networks will also be higher. By using an edge computing-based video surveillance system to migrate video processing to the edge, the computing, storage and network bandwidth requirements of the cloud center can be reduced, which also reduces the requirements for ship satellite networks. It can not only improve the efficiency of video image analysis, but also be more suitable for monitoring videos on ships.

[0003] High Efficiency Video Coding (HEVC) is a new generation of video coding standard, which is widely used in the field of video surveillance. However, the process of dividing the coding tree unit is complicated, and different prediction methods (such as intra-frame prediction and inter-frame prediction) are used for different frames. Using HEVC coding for surveillance video coding, on the one hand, is not adaptable enough and has low compression efficiency. At the same time, there is a difference in computing power between edge computing and cloud computing, so the burden of computing is heavy. In addition, the network bandwidth environment will also affect the efficiency of video coding. When video encoding is performed without considering the influence of the network bandwidth environment, the compression and transmission effect of the encoded video will be affected. On the other hand, due to the use of ship satellite networks on ships, the network bandwidth environment is relatively unstable, which will affect the transmission effect of encoded video. Summary of the invention

[0004] The purpose of this application is to provide a surveillance video intelligent compression method based on ship satellite network edge computing to solve the technical problems of high computational complexity and low compression efficiency in existing surveillance video encoding.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] In the first aspect, the present application proposes a technical solution for a monitoring video intelligent compression method based on ship satellite network edge computing, and the monitoring video intelligent compression method based on ship satellite network edge computing includes:

[0007] Acquire a monitoring video in a ship satellite network application environment; the monitoring video includes at least two time-series images; acquire a network congestion degree according to a network throughput distribution and a network delay before video encoding in the ship satellite network application environment;

[0008] As a specific solution in the technical solution of the present application, the network congestion degree is obtained according to the network throughput distribution and network delay before video encoding in the ship satellite network application environment, including:

[0009]

[0010] in, The preset time period before encoding The degree of network congestion within the country; Indicates the preset time period before encoding Average throughput within Represents the average throughput of the network; Indicates The latency of the edge computing box at each moment; Indicates The latency of the edge computing box at each moment; is the normalization function;

[0011] The surveillance video is divided to obtain at least two data segments; the time sequence of each image in each data segment is adjacent, and the first feature similarity of the adjacent time sequence images is greater than or equal to a first preset value;

[0012] Based on at least two data segments, obtaining a first data segment; the first data segment is any data segment of the at least two data segments;

[0013] Based on the first data segment, a first time-series image and a second time-series image are acquired; the first time-series image and the second time-series image are any two adjacent time-series images in the first data segment;

[0014] Acquire a second feature similarity between the first time-series image and the second time-series image;

[0015] Based on the second feature similarity and the network congestion level, obtaining a division degree; the division degree is at least used to represent the size of the coding tree unit;

[0016] Based on the degree of division, the first data segment is encoded.

[0017] As a specific solution in the technical solution of the present application, the monitoring video is divided to obtain at least two data segments, including:

[0018] Based on the monitoring video, a three-dimensional coordinate system is established; the X-axis of the three-dimensional coordinate system is the time sequence, the Y-axis is the length of the time sequence image, and the Z-axis is the width of the time sequence image;

[0019] Based on the monitoring video, a third time-series image and a fourth time-series image are acquired; the third time-series image and the fourth time-series image are any two adjacent time-series images among at least two time-series images;

[0020] Based on the edge detection algorithm, first point cloud data is obtained from the third time-series image; the first point cloud data at least includes the coordinates and grayscale values ​​of each key point in the third time-series image;

[0021] Based on the edge detection algorithm, second point cloud data is obtained from the fourth time-series image; the second point cloud data at least includes the coordinates and grayscale values ​​of each key point in the fourth time-series image;

[0022] Based on the first point cloud data and the second point cloud data, obtaining a first feature similarity between the third time-series image and the fourth time-series image;

[0023] If the first feature similarity is greater than or equal to a first preset value, the third time-series image and the fourth time-series image are divided into the same data segment.

[0024] As a specific solution in the technical solution of the present application, the obtaining of the first feature similarity between the third time-series image and the fourth time-series image based on the first point cloud data and the second point cloud data includes:

[0025] Based on the first point cloud data, obtaining at least two first point clouds;

[0026] Based on the second point cloud data, obtaining at least two second point clouds;

[0027] Based on at least two first point clouds, obtaining a third point cloud; the third point cloud is any one of the at least two first point clouds;

[0028] Based on the at least two second point clouds, obtaining a fourth point cloud; the fourth point cloud is any one of the at least two second point clouds;

[0029] Based on the third point cloud and the fourth point cloud, obtaining a point cloud matching degree; the point cloud matching degree is at least used to characterize the similarity between the third point cloud and the fourth point cloud;

[0030] Based on the point cloud matching degree, a first feature similarity between the third time-series image and the fourth time-series image is obtained.

[0031] As a specific solution in the technical solution of the present application, the obtaining of the first feature similarity between the third time-series image and the fourth time-series image based on the point cloud matching degree includes:

[0032] Based on the point cloud matching degree, a first quantity is obtained; the first quantity is the quantity of the point clouds in at least two first point clouds and at least two second point clouds whose matching degree is greater than or equal to a second preset value;

[0033] Based on the first point cloud data, obtaining a second quantity; the second quantity is the quantity of the first point cloud in the first point cloud data;

[0034] Based on the second point cloud data, obtaining a third quantity; the third quantity is the quantity of the second point clouds in the second point cloud data;

[0035] Based on the first number, the second number, the third number, and the point cloud matching degree, a first feature similarity between the third time-series image and the fourth time-series image is acquired.

[0036] As a specific solution in the technical solution of the present application, the obtaining of the second feature similarity between the first time-series image and the second time-series image includes:

[0037] Based on the first time-series images, obtaining a fifth point cloud; the fifth point cloud is any point cloud in the first time-series images;

[0038] Based on the second time-series images, a sixth point cloud is acquired; the sixth point cloud is a point cloud in the second time-series images that has the highest matching degree with the fifth point cloud;

[0039] Based on the fifth point cloud and the sixth point cloud, the second feature similarity is acquired.

[0040] As a specific solution in the technical solution of the present application, the obtaining of the second feature similarity based on the fifth point cloud and the sixth point cloud includes:

[0041] Based on the fifth point cloud, obtaining a first convex hull;

[0042] Based on the sixth point cloud, obtaining a second convex hull;

[0043] Based on the first convex hull and the second convex hull, the second feature similarity is obtained.

[0044] As a specific solution in the technical solution of the present application, encoding the first data segment based on the division degree includes:

[0045] Based on the first data segment, a key time sequence image is acquired; the key time sequence image is the image that is the earliest in the first data segment;

[0046] Based on the key time sequence image, obtaining at least two coding tree units;

[0047] Based on at least two coding tree units, a first coding tree unit and a second coding tree unit are obtained; the first coding tree unit is any one of the at least two coding tree units; the second coding tree unit is a coding tree unit different from the first coding tree unit in the at least two coding tree units;

[0048] Acquire a third feature similarity based on the first coding tree unit and the second coding tree unit;

[0049] Based on the third feature similarity, obtaining a coding tree unit set;

[0050] The first data segment is encoded based on the set of coding tree units.

[0051] As a specific solution in the technical solution of the present application, based on the third feature similarity, obtaining a coding tree unit set includes:

[0052] If the third feature similarity is greater than or equal to a third preset value, placing the first coding tree unit and the second coding tree unit into the same coding tree unit set;

[0053] If the third feature similarity is less than a third preset value, placing the second coding tree unit into a new coding tree unit set;

[0054] As a specific solution in the technical solution of this application, the first preset value is set to 0.7;

[0055] As a specific solution in the technical solution of the present application, the second preset value is set to 0.5.

[0056] Compared with the prior art, the beneficial effects of this application are:

[0057] The present application analyzes the changes in adjacent time-series images of surveillance videos, performs adaptive segmentation, establishes segmented three-dimensional models, analyzes the changes in feature points of the segments, and completes the compression of segmented videos. While improving the compression efficiency of video encoding, it also reduces the amount of edge computing computation, and combines the network bandwidth environment for analysis, making it suitable for intelligent video encoding and compression for edge computing. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of the structure of a three-dimensional model proposed in an embodiment of the present application;

[0059] Figure 2 A flowchart of a monitoring video intelligent compression method based on ship satellite network edge computing proposed in an embodiment of the present application;

[0060] Figure 3The figure is a schematic diagram of the HEVC encoding process in the prior art. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] The terms "first", "second", etc. in the description of the embodiment of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, such as the first preset value and the second preset value proposed below, which belong to different preset values. It should be understood that the preset values ​​used in this way can be interchanged where appropriate, so that the embodiments described here can be implemented in an order other than the content illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device including a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The division of modules that appear in the embodiment of the present application is only a logical division. There may be other division methods when implemented in actual applications, such as at least two modules can be combined or integrated into another system, or some features can be ignored, or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in the embodiment of the present application. Furthermore, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in at least two circuit modules, and some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0063] In order to solve the technical problems raised by the background technology, this application proposes an embodiment of a monitoring video intelligent compression method based on ship satellite network edge computing. For details, please refer to Figure 2 , which shows a flow chart of a monitoring video intelligent compression method based on ship satellite network edge computing proposed in an embodiment of the present application, the method comprising steps S100 to S700.

[0064] Step S100: Acquire a monitoring video in a ship satellite network application environment; the monitoring video includes at least two time-series images; and acquire the degree of network congestion according to the network throughput distribution and network delay before video encoding in the ship satellite network application environment.

[0065] It should be clear that in the embodiments of the present application, the surveillance video can be a surveillance video of any occasion. However, as will be seen below, the embodiments of the present application are more effective for surveillance videos with small changes over a long period of time, such as surveillance videos in ships, surveillance videos in underground parking garages, or surveillance videos in warehouses. The subsequent embodiments of the present application are only described with reference to surveillance videos in ships, which does not mean that the embodiments proposed in the present application can only be applied to the application scenario of surveillance videos in ships. It is easy to understand that the surveillance videos are composed of at least two time-series images, that is, the surveillance videos described in the embodiments of the present application include at least two time-series images.

[0066] Specifically, this application takes the monitoring in the ship's wheelhouse as an example and adopts the HEVC encoding method for encoding. Unlike media videos, etc., monitoring videos have a strong repetitive nature, and HEVC encoding removes redundant information in the time and space domains of consecutive frames respectively, and the encoding efficiency is low. For the monitoring in the ship's wheelhouse, the status of the crew, the status of the machines related to the driving area, etc. are relatively important, and these are in relative motion. Therefore, for the continuous frames of the monitoring video, the change of the pixel points also represents the importance of the corresponding area. If the continuous frames (that is, each time-series image) change less, then there are more images that can be encoded as a whole. In other words, this embodiment can preliminarily segment the video according to the change of the pixel points of the continuous frames. The change of the pixel points of the continuous frames also represents the importance of the corresponding image partition, and the division process of the HEVC encoding can also be divided accordingly. Next, the video segmentation is taken as a whole, and the change of the pixel points of the continuous frames is represented as a matrix, and used as the data of the HEVC encoding quantization process.

[0067] In the embodiment of the present application, the degree of network congestion is obtained according to the network throughput distribution and network delay before video encoding in the ship satellite network application environment, including:

[0068]

[0069] in, The preset time period before encoding The degree of network congestion within the country; Indicates the preset time period before encoding Average throughput within Represents the average throughput of the network; Indicates The latency of the edge computing box at each moment; Indicates The latency of the edge computing box at each moment; is the normalization function. The network throughput before video encoding is subtracted from the average throughput of all time, indicating the network congestion before encoding. The larger the value, the more congested it is. It means adding the delay difference of each adjacent moment to indicate the growth of the delay. The larger the value, the more significant the delay increase, and thus the greater the network congestion. In the embodiment of the present application, the preset time period is set to a half-hour time period. That is, the degree of network congestion within half an hour before encoding. The preset time period can be adjusted according to the specific implementation environment. Length.

[0070] Step S200: Divide the surveillance video into at least two data segments.

[0071] In an embodiment of the present application, the surveillance video can be divided in any suitable manner to obtain at least two data segments. As can be seen from the foregoing, the time-series images in the same data segment are adjacent in sequence and have similar features. That is to say, in an embodiment of the present application, at least two data segments can be divided from the surveillance video according to the timing and similarity of the time-series images. That is to say, in an embodiment of the present application, the timing of the images in each data segment is adjacent, and the first feature similarity of the adjacent time-series images is greater than or equal to a first preset value.

[0072] In the embodiment of the present application, the first preset value can be set according to the requirements (the second preset value and the third preset value below are also the same, and will not be described in detail later). In a specific implementation of the embodiment of the present invention, the first preset value is set to 0.7, and the implementer can set it to 0.8 or 0.9 according to the specific implementation environment. It is easy to understand that if the first preset value is larger, the more data segments are divided from the same monitoring video; if the first preset value is smaller, the fewer data segments are divided from the same monitoring video.

[0073] In a specific embodiment of the present application, step S200, dividing the surveillance video to obtain at least two data segments, includes steps S210 to S260.

[0074] Step S210: Establishing a three-dimensional coordinate system based on the surveillance video.

[0075] In the embodiment of the present application, the X-axis of the three-dimensional coordinate system is the time sequence, the Y-axis is the length of the time sequence image, and the Z-axis is the width of the time sequence image. Figure 1, which shows a schematic diagram of the structure of a three-dimensional model proposed in an embodiment of the present application, where the X-axis is time, and any time-series image of the monitoring video is parallel to the YOZ plane in the three-dimensional coordinate system, and the coordinates of the first pixel point in the lower right corner of the first frame image (that is, the first time-series image) are The distance between consecutive frames is 1 pixel, and the Y-axis and Z-axis represent the horizontal and vertical coordinates of the image pixels respectively.

[0076] Step S220: Based on the monitoring video, a third time-series image and a fourth time-series image are acquired.

[0077] In this embodiment, the third time sequence image and the fourth time sequence image are any two adjacent time sequence images of at least two time sequence images. That is to say, in this embodiment, any two adjacent time sequence images need to be compared for similarity later, and if the similarity exceeds a first preset value, the two time sequence images can be divided into the same data segment.

[0078] In the embodiment of the present application, the similarity between two time-series images may be calculated in any manner, such as: Euclidean distance algorithm or Manhattan distance algorithm, etc. Alternatively, in the embodiment of the present application, the similarity between two time-series images is calculated based on the edge detection algorithm described below.

[0079] Step S230: Based on an edge detection algorithm, first point cloud data is acquired from the third time-series image.

[0080] In this embodiment, an edge detection algorithm is used to obtain a plurality of edges of the third time series image, and a plurality of edges are uniformly selected from any edge (for example, it can be indivual, is the number of edge pixels, The pixel point ( ) is taken as the key point of the edge, all the key points of the third time series image are taken as the first point cloud data, the adjacent pixel points when the pixel points are uniformly taken are recorded as the adjacent point cloud, and the coordinates are the point cloud coordinates. The first point cloud data at least includes the coordinates and grayscale values ​​of each key point in the third time series image.

[0081] It should be clear that the edge detection algorithm is a mature technology and will not be described in detail here.

[0082] Step S240: Based on an edge detection algorithm, second point cloud data is acquired from the fourth time-series image.

[0083] It should be noted that the method for acquiring the second point cloud data is the same as step S230, which will not be described in detail herein. The second point cloud data at least includes the coordinates and grayscale values ​​of each key point in the fourth time-series image.

[0084] Step S250: Based on the first point cloud data and the second point cloud data, obtaining a first feature similarity between the third time-series image and the fourth time-series image.

[0085] It should be clear that in the embodiment of the present application, the first feature similarity between the third time-series image and the fourth time-series image can be obtained in any reasonable manner, for example, the Euclidean distance algorithm or the Manhattan distance algorithm mentioned above.

[0086] In order to reduce the amount of calculation, in one embodiment of the present application, step S250, based on the first point cloud data and the second point cloud data, obtains the first feature similarity of the third time-series image and the fourth time-series image, including steps S251 to S256.

[0087] Step S251: Based on the first point cloud data, obtain at least two first point clouds.

[0088] It should be noted that obtaining point clouds based on point cloud data is a mature technology and will not be elaborated here.

[0089] Step S252: Based on the second point cloud data, obtain at least two second point clouds.

[0090] It should be noted that obtaining point clouds based on point cloud data is a mature technology and will not be elaborated here.

[0091] Step S253: Based on at least two first point clouds, obtain a third point cloud.

[0092] In this embodiment, the third point cloud is any one of the at least two first point clouds.

[0093] Step S254: Based on at least two second point clouds, obtain a fourth point cloud.

[0094] In this embodiment, the fourth point cloud is any one of the at least two second point clouds.

[0095] Step S255: Based on the third point cloud and the fourth point cloud, obtain the point cloud matching degree.

[0096] That is to say, in this embodiment, each first point cloud and second point cloud need to be paired one by one to calculate the similarity, so as to determine whether a first point cloud (that is, the third point cloud) corresponds to a second point cloud (that is, the fourth point cloud) in different time-series images. That is to say, in this embodiment, the point cloud matching degree is at least used to characterize the similarity between the third point cloud and the fourth point cloud.

[0097] Specifically, in this embodiment, in step S255, based on the third point cloud and the fourth point cloud, a calculation formula for obtaining the point cloud matching degree is as follows:

[0098]

[0099] in, Indicates the point cloud matching degree between the third point cloud and the fourth point cloud; Represents the gray value of the third point cloud; Represents the gray value of the fourth point cloud; Indicates the absolute value. represents an exponential function with a natural constant as the base; A is the number of pixels in the third point cloud; is a self-defined function, indicating the consistency of the a-th pixel in the third point cloud and the fourth point cloud as point cloud data. If both are point clouds or neither is a point cloud, then , if one is a point cloud and the other is not a point cloud, then ; It is a normalization function, which is used to map the values ​​in brackets to the interval (0, 1).

[0100] In this embodiment, Represents the absolute value of the grayscale value difference between the third point cloud and the fourth point cloud. The larger the value, the greater the difference between the two point clouds, and the smaller the possibility of matching point clouds; Indicates the consistency of the positions of the third point cloud and the fourth point cloud. If all positions are point clouds, the possibility that the two point clouds are matching point clouds is greater.

[0101] Step S256: Based on the point cloud matching degree, obtaining the first feature similarity between the third time-series image and the fourth time-series image.

[0102] It should be clear that in the embodiments of the present application, the average value of the matching degree of each point cloud can be used as the first feature similarity. It is easy to understand that if the point cloud matching degree of each point cloud in the third time sequence image and the fourth time sequence image is higher, it means that the similarity between the third time sequence image and the fourth time sequence image is also higher.

[0103] In one embodiment of the present application, step S256, based on the point cloud matching degree, obtaining the first feature similarity of the third time-series image and the fourth time-series image, includes steps S261 to S264.

[0104] Step S261: Based on the point cloud matching degree, obtain a first quantity.

[0105] In an embodiment of the present application, the first number is the number of point clouds in at least two first point clouds and at least two second point clouds whose matching degree is greater than or equal to a second preset value.

[0106] In this embodiment, the second preset value can be set to any suitable value according to requirements. In a specific implementation of the embodiment of the present invention, the second preset value is set to 0.5, and the implementer can set it to 0.6 or 0.7 according to the specific implementation environment.

[0107] Step S262: Obtain a second quantity based on the first point cloud data.

[0108] In an embodiment of the present application, the second number is the number of first point clouds in the first point cloud data.

[0109] Step S263: Based on the second point cloud data, obtain a third quantity.

[0110] In an embodiment of the present application, the third number is the number of second point clouds in the second point cloud data.

[0111] Step S264: Based on the first number, the second number, the third number and the point cloud matching degree, obtain the first feature similarity between the third time-series image and the fourth time-series image.

[0112] It should be noted that, if there are more similar point clouds, the similarity of the first feature between the third time-series image and the fourth time-series image will be greater.

[0113] In a specific embodiment of the present application, in step S264, based on the first number, the second number, the third number and the point cloud matching degree, a calculation formula for obtaining the first feature similarity of the third time-series image and the fourth time-series image is as follows:

[0114]

[0115] in, Indicates the first feature similarity between the third time series image and the fourth time series image; represents a first quantity; represents a second quantity; Indicates the third quantity; Represents the point cloud matching degree of the qth matching point cloud; It is a normalization function, which is used to map the values ​​in brackets to the interval (0, 1).

[0116] In this embodiment, Indicates the proportion of matching point clouds in the first point cloud data and the second point cloud data (that is, point clouds with a point cloud matching degree greater than or equal to the second preset value) in their respective point clouds; the larger the value, the more matching point clouds in the first point cloud data and the second point cloud data, that is, the more similar the third time series image and the fourth time series image are. It means adding the matching degrees of all matching point clouds. The larger the value is, the greater the matching degree is, that is, the more similar the third time series image is to the fourth time series image.

[0117] Step S260: if the first feature similarity is greater than or equal to a first preset value, dividing the third time-series image and the fourth time-series image into the same data segment.

[0118] In an embodiment of the present application, time-series images with a first feature similarity greater than or equal to a first preset value may be divided into the same data segment. It is easy to understand that if the first feature similarity is less than the first preset value, the fourth time-series image is used as the first time-series image in the next data segment.

[0119] Step S300: Based on at least two data segments, obtain a first data segment.

[0120] It should be clear that when HEVC encoding divides the coding tree units, it relies on the richness of the information therein. However, for a cabin that already contains a lot of information, relying solely on the richness of information is not entirely applicable. Therefore, the moving area should be divided more finely, while the coding tree units of the areas that have not changed can be relatively larger to save the amount of calculation for subsequent encoding and improve compression efficiency. It is easy to understand that all data segments need to be encoded, so in this embodiment, the first data segment is any data segment of at least two data segments.

[0121] Step S400: Based on the first data segment, a first time-series image and a second time-series image are acquired.

[0122] As can be seen from the foregoing, in the application scenario of cabin surveillance video, in most time periods, the objects in each time-series image (for example, power distribution cabinets, life buoys and other equipment) are stationary. In the subsequent encoding process, the coding tree units of these time-series images can be made larger. If the objects in the time-series images (for example, people or animals, etc.) are moving, in the subsequent encoding process, the coding tree units of these time-series images can be made smaller. That is to say, in this embodiment, how to judge whether the objects in each time-series image are stationary or moving is of paramount importance. In this embodiment, the first time-series image and the second time-series image are any two adjacent time-series images in the first data segment.

[0123] Step S500: obtaining a second feature similarity between the first time-series image and the second time-series image.

[0124] It should be clear that in an embodiment of the present application, the second feature similarity of the first time-series image and the second time-series image can be calculated in the manner described above. In order to enable the subsequent determination of whether the objects in the two time-series images move based on the second feature similarity, in an embodiment of the present application, step S500, obtaining the second feature similarity of the first time-series image and the second time-series image, includes steps S510 to S530.

[0125] Step S510: Acquire a fifth point cloud based on the first time-series images.

[0126] In this embodiment, the fifth point cloud is any point cloud in the first time-series image.

[0127] Step S520: Acquire a sixth point cloud based on the second time-series images.

[0128] In this embodiment, the sixth point cloud is the point cloud in the second time-series image that has the highest matching degree with the fifth point cloud.

[0129] Step S530: Based on the fifth point cloud and the sixth point cloud, obtain the second feature similarity.

[0130] It should be clear that if an object moves, then in the time-series images, the point clouds formed by the object are not only relatively similar in each time-series image, but the corresponding coordinates will also change. In other words, in the embodiments of the present application, it is possible to determine whether the object corresponding to the point cloud has moved based on the shapes, coordinates and similarities corresponding to the two point clouds.

[0131] Specifically, step S530, based on the fifth point cloud and the sixth point cloud, obtains the second feature similarity, including steps S531 to S533.

[0132] Step S531: Based on the fifth point cloud, obtain a first convex hull.

[0133] It should be clear that the convex hull is a concept in computational geometry (graphics), which is a mature technology and will not be described in detail here. It should be noted that obtaining the corresponding convex hull based on the point cloud is also a mature technology and will not be described in detail here.

[0134] Step S532: Based on the sixth point cloud, obtain a second convex hull.

[0135] It should be noted that obtaining the corresponding convex hull based on the point cloud is a mature technology and will not be described in detail here.

[0136] Step S533: Based on the first convex hull and the second convex hull, obtain the second feature similarity.

[0137] In this embodiment, the second feature similarity can be obtained based on the first convex hull and the second convex hull in any appropriate manner. For example, the second feature similarity can be obtained by using the Euclidean distance algorithm or the Manhattan distance algorithm described above.

[0138] In a specific embodiment of the present application, step S533: based on the first convex hull and the second convex hull, a calculation formula for obtaining the second feature similarity is as follows:

[0139]

[0140] in, Indicates the similarity of the second feature, that is, it is used to indicate the possibility that the fifth point cloud and the sixth point cloud belong to the motion point cloud; Indicates the number of time-series images in the first data segment; Represents the convex hull volume of all matching point clouds; Represents the mth element value in the residual matrix of the fifth and sixth point clouds, Represents the total number of elements in the residual matrix of the fifth and sixth point clouds.

[0141] It should be noted that the elements of the first row are subtracted from the elements of any row of the matching point cloud coordinate matrix of any point cloud in the first data segment, and recorded as the matching point cloud residual matrix of the point cloud.

[0142] In this embodiment, Indicates the matching situation in the first data segment. The larger V is, the more similar the time-series images in the video segment are. Therefore, the possibility that the point cloud in each time-series image belongs to the motion point cloud is smaller. The smaller the area, the more concentrated the projection points are, the smaller the degree of movement of the point cloud is, and the smaller the possibility of being a moving point cloud is. It means adding all the element values ​​of the matching point cloud residual matrix, which shows the change of the matching point cloud sequence and also indicates the degree of movement of the point cloud. The smaller the value, the smaller the possibility of belonging to the moving point cloud.

[0143] In this embodiment, point clouds with a probability of belonging to motion point clouds greater than 0.6 are recorded as motion point clouds; all motion point clouds in any key frame are used as seed points, and the number of motion point clouds in the seed point window is greater than 1 as a condition for seed point growth, and regional growth is performed. When all motion point clouds in the region cannot continue to grow, the regional growth is terminated; a number of growth regions are obtained; if the growth region is an annular region, the coding tree units contained in the growth region and the coding tree units in the annular region together constitute one motion region; if the growth region is not an annular region, the coding tree units contained in the growth region constitute one motion region; (motion regions with overlapping parts are merged into one motion region).

[0144] Step S600: Acquire a division degree based on the second feature similarity and the network congestion degree.

[0145] It should be clear that, as can be seen from the foregoing, if the second feature similarity is smaller, the possibility that the point cloud corresponding to each time-series image in the first data segment belongs to the motion point cloud is smaller, that is, when dividing the coding tree unit, the larger the division can be. If the second feature similarity is larger, the possibility that the point cloud corresponding to each time-series image in the first data segment belongs to the motion point cloud is larger, that is, when dividing the coding tree unit, the smaller the division can be. That is to say, in this embodiment, the division degree is at least used to represent the size of the coding tree unit.

[0146] In one embodiment of the present application, step S600: based on the second feature similarity and the network congestion level, a calculation formula for obtaining the division level is as follows:

[0147]

[0148] in, Indicates the degree of division; Indicates the motion coefficient. If the coding tree unit is a motion area, there is a motion coefficient, and K=1.5; Indicates the number of grayscale intervals contained in the grayscale-pixel histogram of the coding tree unit; Indicates the number of pixels in the grayscale interval with the largest number of pixels in the grayscale-pixel histogram of the coding tree unit; Represents the number of pixels in the bth grayscale interval in the grayscale-pixel histogram of the coding tree unit; is a normalization function, which is used to map the values ​​in brackets to the interval (0, 1); represents an exponential function with a natural constant as base; The preset time period before encoding The greater the network congestion, the smaller the division should be, so that the compression is more sufficient.

[0149] It should be noted that in this embodiment, for any coding tree unit, 8 gray levels are used as an interval, and the number of pixels in each gray level interval is obtained to form a gray level-pixel histogram; the coding tree unit size of HEVC coding is .

[0150] In this embodiment, there should be a greater degree of division for the motion area to ensure video quality; It represents the sum of the differences between the number of pixels in the grayscale interval with the largest number of pixels and the number of pixels in all other intervals. The larger the value, the more concentrated the grayscale is and the less information is contained. The wider the grayscale range of the coding tree unit and the more dispersed the grayscale is, the higher the degree of division of the coding tree unit is.

[0151] In this embodiment, any coding tree unit is divided according to the division degree of the coding tree unit: , then the coding tree unit maintains ;like , then the coding tree unit is divided into 4 subunit; if , then the coding tree unit is divided into 16 subunit; if , then the coding tree unit is divided into 64 subunit of .

[0152] Step S700: Encode the first data segment based on the division degree.

[0153] It should be clear that, in the embodiments of the present application, the first data segment can be directly encoded based on HEVC encoding.

[0154] It should be noted that the HEVC encoding flow chart is as follows Figure 3 As shown in the figure, each frame of the time series image is divided into a motion area and a non-motion area. For an environment such as cabin monitoring, there are fewer motion areas in the image and more non-motion areas. Therefore, there will be a large number of zeros in the residual grayscale matrix of the pixel points of the continuous frame image of the video segmentation. The residual matrix can be considered as a sparse matrix. Therefore, for the non-key frames of the video segmentation, the residual matrix can be transferred to the quantization step; for the key frames (that is, the time series images sorted first in each data segment), since there is spatial redundancy in the background, the key frames can be further analyzed to simplify the quantization information of the key frames.

[0155] In order to further simplify the quantization information of the key frame, step S700, based on the division degree, encodes the first data segment, including steps S710 to S760.

[0156] Step S710: Acquire a key time sequence image based on the first data segment.

[0157] In this embodiment, the key time sequence image is the image that is the earliest in time sequence in the first data segment.

[0158] Step S720: Based on the key temporal image, obtain at least two coding tree units.

[0159] It should be noted that, in this embodiment, the at least two coding tree units are based on the at least two coding tree units obtained in step S600.

[0160] Step S730: Based on at least two coding tree units, obtain a first coding tree unit and a second coding tree unit.

[0161] In this embodiment, the first coding tree unit is any one of at least two coding tree units; and the second coding tree unit is a coding tree unit different from the first coding tree unit among the at least two coding tree units.

[0162] Step S740: Obtain a third feature similarity based on the first coding tree unit and the second coding tree unit.

[0163] In the embodiment of the present application, any suitable algorithm may be used to obtain the third feature similarity based on the first coding tree unit and the second coding tree unit, for example, the Euclidean distance algorithm or the Manhattan distance algorithm mentioned above.

[0164] In one embodiment of the present application, in step S740, based on the first coding tree unit and the second coding tree unit, a calculation formula for obtaining a third feature similarity is as follows:

[0165]

[0166] in, Indicates a third feature similarity between the first coding tree unit and the second coding tree unit; Indicates the division degree corresponding to the first coding tree unit; Indicates the division degree corresponding to the second coding tree unit; Indicates the number of pixels in the coding tree unit; Represents the grayscale of the nth pixel in the first coding tree unit; Represents the grayscale of the nth pixel in the second coding tree unit; represents an exponential function with a natural constant as base; The smaller the difference in the degree of division and the smaller the difference in the grayscale of all corresponding pixels, the higher the similarity of the third feature between the first coding tree unit and the second coding tree unit.

[0167] Step S750: Based on the third feature similarity, obtain a coding tree unit set.

[0168] It should be understood that, in the embodiments of the present application, coding tree units with greater similarity may be placed into the same coding tree unit set.

[0169] Specifically, step S750, based on the third feature similarity, obtains a coding tree unit set, including step S751 and step S752.

[0170] Step S751: If the third feature similarity is greater than or equal to a third preset value, placing the first coding tree unit and the second coding tree unit into the same coding tree unit set.

[0171] In this embodiment, the third preset value can select any appropriate value. In a specific implementation of the embodiment of the present invention, the third preset value is set to 0.7; it can also be set to 0.8 or 0.9 according to the specific implementation environment.

[0172] Step S752: If the third feature similarity is less than a third preset value, placing the second coding tree unit into a new coding tree unit set.

[0173] Step S760: Encode the first data segment based on the coding tree unit set.

[0174] In an embodiment of the present application, for any similar coding tree unit set, a key coding tree unit grayscale matrix is ​​obtained, and the residual matrix (all element grayscale values ​​are subtracted) between the remaining coding tree unit grayscale matrices of the similar coding tree unit set and the key coding tree unit grayscale matrix is ​​calculated to obtain several residual matrices of any key frame, which are recorded as a type-one residual matrix. For any non-key frame of a data segment (that is, other time-series images that are not in the front time sequence in the data segment), a residual matrix between the total grayscale matrix and the total grayscale matrix of the key frame is obtained, which is recorded as a type-two residual matrix. The type-one residual matrix and the type-two residual matrix are transformed / quantized, and subsequently entropy encoded to obtain an output bitstream, thereby completing the encoding of the video segment.

[0175] It should be noted that encoding the first data segment is a mature technology and will not be described in detail here.

[0176] It should be clear that the embodiment of the monitoring video intelligent compression method based on ship satellite network edge computing proposed in this application analyzes the changes in adjacent time-series images of the monitoring video, performs adaptive segmentation, establishes a segmented three-dimensional model, analyzes the changes in the feature points of the segment, completes the compression of the segmented video, improves the compression efficiency of video encoding, and reduces the computational complexity of edge computing, making it suitable for intelligent video encoding for edge computing.

[0177] Although the embodiments of the present application have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles of the present application.

Claims

1. A monitoring video intelligent compression method based on ship satellite network edge computing, characterized in that: include: Acquire surveillance video in the ship satellite network application environment; The surveillance video includes at least two time-series images; According to the network throughput distribution and network delay before video encoding in the ship satellite network application environment, the network congestion degree is obtained; The surveillance video is divided to obtain at least two data segments; the time sequence of each image in each data segment is adjacent, and the first feature similarity of the adjacent time sequence images is greater than or equal to a first preset value; The arbitrary time-series images of the surveillance video are parallel to the YOZ plane in the three-dimensional coordinate system, and the coordinates of the first pixel point in the lower right corner of the first frame image are taken as , the distance between consecutive frames is 1 pixel, and the Y axis and Z axis represent the horizontal and vertical coordinates of the image pixel points respectively; Acquire point cloud data from the time-series images, and acquire first feature similarity of the time-series images based on the point cloud data; Based on at least two data segments, obtaining a first data segment; The first data segment is any data segment of at least two data segments; Based on the first data segment, a first time-series image and a second time-series image are acquired; the first time-series image and the second time-series image are any two adjacent time-series images in the first data segment; Acquire a second feature similarity between the first time-series image and the second time-series image; Based on the second feature similarity and the network congestion level, obtaining a division degree; the division degree is at least used to represent the size of the coding tree unit; Based on the degree of division, the first data segment is encoded.

2. The monitoring video intelligent compression method based on ship satellite network edge computing according to claim 1 is characterized in that: The obtaining of the network congestion degree according to the network throughput distribution and network delay before video encoding in the ship satellite network application environment includes: in, The preset time period before encoding The degree of network congestion within the Indicates the preset time period before encoding Average throughput within Represents the average throughput of the network; Indicates The latency of the edge computing box at each moment; Indicates The latency of the edge computing box at each moment; is the normalization function.

3. The monitoring video intelligent compression method based on ship satellite network edge computing according to claim 1 is characterized in that: The step of dividing the surveillance video to obtain at least two data segments includes: Based on the monitoring video, a three-dimensional coordinate system is established; the X-axis of the three-dimensional coordinate system is the time sequence, the Y-axis is the length of the time sequence image, and the Z-axis is the width of the time sequence image; based on the monitoring video, a third time sequence image and a fourth time sequence image are obtained; the third time sequence image and the fourth time sequence image are any two adjacent time sequence images of at least two time sequence images; Based on the edge detection algorithm, first point cloud data is obtained from the third time-series image; the first point cloud data at least includes the coordinates and grayscale values ​​of each key point in the third time-series image; Based on the edge detection algorithm, second point cloud data is obtained from the fourth time-series image; the second point cloud data at least includes the coordinates and grayscale values ​​of each key point in the fourth time-series image; Based on the first point cloud data and the second point cloud data, obtaining a first feature similarity between the third time-series image and the fourth time-series image; If the first feature similarity is greater than or equal to a first preset value, the third time-series image and the fourth time-series image are divided into the same data segment.

4. The monitoring video intelligent compression method based on ship satellite network edge computing according to claim 3 is characterized in that: The acquiring, based on the first point cloud data and the second point cloud data, a first feature similarity between the third time-series image and the fourth time-series image comprises: Based on the first point cloud data, obtaining at least two first point clouds; Based on the second point cloud data, obtaining at least two second point clouds; Based on at least two first point clouds, obtaining a third point cloud; the third point cloud is any one of the at least two first point clouds; Based on the at least two second point clouds, obtaining a fourth point cloud; the fourth point cloud is any one of the at least two second point clouds; Based on the third point cloud and the fourth point cloud, obtaining a point cloud matching degree; the point cloud matching degree is at least used to characterize the similarity between the third point cloud and the fourth point cloud; Based on the point cloud matching degree, a first feature similarity between the third time-series image and the fourth time-series image is obtained.

5. The monitoring video intelligent compression method based on ship satellite network edge computing according to claim 4 is characterized in that: The acquiring, based on the point cloud matching degree, a first feature similarity between the third time-series image and the fourth time-series image comprises: Based on the point cloud matching degree, a first quantity is obtained; the first quantity is the quantity of the point clouds in at least two first point clouds and at least two second point clouds whose matching degree is greater than or equal to a second preset value; Based on the first point cloud data, obtaining a second quantity; the second quantity is the quantity of the first point cloud in the first point cloud data; Based on the second point cloud data, obtaining a third quantity; the third quantity is the quantity of the second point clouds in the second point cloud data; Based on the first number, the second number, the third number, and the point cloud matching degree, a first feature similarity between the third time-series image and the fourth time-series image is acquired.

6. The monitoring video intelligent compression method based on ship satellite network edge computing according to claim 4 is characterized in that: The obtaining the second feature similarity between the first time-series image and the second time-series image includes: Based on the first time-series images, obtaining a fifth point cloud; the fifth point cloud is any point cloud in the first time-series images; Based on the second time-series images, a sixth point cloud is acquired; the sixth point cloud is a point cloud in the second time-series images that has the highest matching degree with the fifth point cloud; Based on the fifth point cloud, obtaining a first convex hull; Based on the sixth point cloud, obtaining a second convex hull; Based on the first convex hull and the second convex hull, the second feature similarity is obtained.

7. The monitoring video intelligent compression method based on ship satellite network edge computing according to any one of claims 1 to 6, characterized in that: The encoding of the first data segment based on the division degree includes: Based on the first data segment, a key time sequence image is acquired; the key time sequence image is the image that is the earliest in the first data segment; Based on the key time sequence image, obtaining at least two coding tree units; Based on at least two coding tree units, a first coding tree unit and a second coding tree unit are obtained; the first coding tree unit is any one of the at least two coding tree units; the second coding tree unit is a coding tree unit different from the first coding tree unit in the at least two coding tree units; Acquire a third feature similarity based on the first coding tree unit and the second coding tree unit; Based on the third feature similarity, obtaining a coding tree unit set; The first data segment is encoded based on the set of coding tree units.

8. The monitoring video intelligent compression method based on ship satellite network edge computing according to claim 7 is characterized in that: Based on the third feature similarity, obtaining a coding tree unit set includes: If the third feature similarity is greater than or equal to a third preset value, placing the first coding tree unit and the second coding tree unit into the same coding tree unit set; If the third feature similarity is less than a third preset value, the second coding tree unit is placed in a new coding tree unit set.

9. The monitoring video intelligent compression method based on ship satellite network edge computing according to claim 1 is characterized in that: The first preset value is set to 0.

7.

10. The monitoring video intelligent compression method based on ship satellite network edge computing according to claim 5 is characterized in that: The second preset value is set to 0.5.

Images