Security monitoring video optimization transmission method and system based on Internet of Things
By analyzing the camera rotation parameters and target behavior in the security surveillance video, filtering out the abnormal behavior target frame images for fidelity transmission, solving the problem of inefficient transmission efficiency of the Internet of Things and improving the transmission efficiency and compression rate of the security surveillance video.
Patent Information
- Application Number
- CN202411319936.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-23
AI Technical Summary
The existing security surveillance videos are inefficient in transmission during the Internet of Things transmission, resulting in the impact of security prevention effects, and lossless compression has no practical significance for the transmission of non-important video frames.
By obtaining the real-time rotation parameters of the camera of the security monitoring video frame image, the target frame image is extracted, the monitoring rotation factor, the movement feature factor and the behavior deviation factor are calculated, the abnormal behavior target frame image is filtered for fidelity transmission, and the non-target frame image is compressed and transmitted.
It improves the transmission efficiency and compression rate of security monitoring videos, ensures the fidelity transmission of abnormal behavior targets, and reduces the storage and transmission burden of normal behavior targets.
Smart Images

Figure CN118869669B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image communication, and in particular to a security monitoring video optimization transmission method and system based on the Internet of Things. Background Art
[0002] As a real-time video surveillance system, the security monitoring system is usually used in sparsely populated public places such as the perimeter of a residential area and blind spots on urban roads to monitor and defend targets within the monitoring range. The cameras in the security monitoring system are usually deployed through a pan / tilt system and have certain target tracking and trajectory prediction functions. By connecting the security monitoring system to the Internet of Things, it is possible to monitor blind spots and sparsely populated public places without scene and time restrictions. However, the security monitoring videos generated by the security monitoring system occupy a huge storage space, and the transmission efficiency is seriously reduced during the transmission process of the Internet of Things, which in turn affects the security prevention effect of the security monitoring system based on the Internet of Things. Therefore, the optimization transmission problem of security monitoring videos needs to be solved urgently.
[0003] In the transmission process of the existing security monitoring video in the Internet of Things, the security monitoring video is usually losslessly compressed and transmitted; since the monitoring area of the security monitoring system is sparsely populated and usually no targets appear, this part of the security monitoring video is actually not important in the security monitoring system, so its lossless compression transmission does not have much practical significance for the security monitoring system; at the same time, the main purpose of the security monitoring system is to identify and track targets with abnormal behavior, and in the target identification process, there are targets with normal behavior and targets with abnormal behavior. The targets with abnormal behavior need to be transmitted with fidelity, thereby ensuring the working efficiency of the security monitoring system and the transmission efficiency of the security monitoring video. Summary of the invention
[0004] The present invention provides a security monitoring video optimization transmission method and system based on the Internet of Things to solve the problem of low transmission efficiency caused by failure to consider target behavior during the existing security monitoring video transmission process. The technical solutions adopted are as follows:
[0005] The present invention proposes a security monitoring video optimization transmission method based on the Internet of Things, which comprises the following steps:
[0006] Obtain security surveillance video and use the PTZ to record the real-time rotation parameters of the camera in each video frame;
[0007] Extracting a number of target frame images from the security monitoring video to obtain a number of target frame sequences; obtaining a monitoring rotation factor for each target frame image according to the real-time rotation parameter changes of the camera under adjacent target frame images in the target frame sequence;
[0008] According to the change of the area ratio of the target area in the adjacent target frame images, the moving characteristic factor of the target area in each target frame image is obtained; based on the position change of the target area in the adjacent target frame images, the moving characteristic factor is adjusted, and combined with the real-time rotation parameter change of the camera, the moving distance of the target area in each target frame image is obtained;
[0009] According to the target area in each target frame image in the target frame sequence and its moving distance, the overall displacement of the target area in the target frame sequence is obtained; according to the centroid position change and grayscale change of the target area in the adjacent target frame images, combined with the moving characteristic factor of the target area in the target frame image, the behavior deviation factor of the target area in each target frame image is obtained; combined with the monitoring rotation factor of each target frame image, the abnormal behavior index of the target area is obtained by comparing the difference between the overall displacement of the target area in the target frame sequence and the moving distance of the target area in each target frame image;
[0010] According to the abnormal behavior indicators of the target area, the target frame images are screened and combined with the non-target frame images for compression and transmission.
[0011] Optionally, the obtaining of the monitoring rotation factor of each target frame image includes the following specific methods:
[0012] For any target frame image, according to the real-time rotation parameters of the camera under the target frame image and the adjacent previous target frame image, the angle change of the target frame image is obtained through the pitch angle change and the steering angle change; the angle change of the target frame image is compared with The ratio of is recorded as the angle change factor of the target frame image;
[0013] The reference number of target frame images before and after the target frame image are used as the reference frame images of the target frame image. The calculation method of the monitoring rotation factor of the target frame image is:
[0014]
[0015] in, Indicates The monitoring rotation factor of the target frame image, Indicates The number of reference frame images of the target frame image, Indicates The angle change factor of the target frame image, Indicates The target frame image Angular variation factor of the reference frame image.
[0016] Optionally, the specific method for obtaining the moving characteristic factor of the target area in each target frame image is as follows:
[0017] Obtain the bounding rectangle of the corresponding area of the target area in each target frame image, recorded as the bounding rectangle of the target area in each target frame image. The calculation method of the moving characteristic factor of the target area in the target frame image is:
[0018]
[0019] in, Indicates The moving characteristic factor of the target area in the target frame image, Indicates the length of the video frame image. and Respectively represent The target frame image and the The length of the bounding rectangle of the target area in the target frame image, Indicates the width of the video frame image. and Respectively represent The target frame image and the The width of the bounding rectangle of the target area in the target frame image.
[0020] Optionally, the obtaining of the moving distance of the target area in each target frame image includes the following specific methods:
[0021] Obtain the centroid of the corresponding area of the target area in each target frame image, construct a coordinate system with the lower left corner of each target frame image as the coordinate origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical upward direction as the positive direction of the vertical axis, and obtain the coordinates of each centroid; take the Euclidean distance between the centroids of the corresponding areas of the target area in adjacent target frame images as the lateral movement distance of the target area in the previous target frame image in the adjacent target frame image;
[0022] No. The calculation method of the moving distance of the target area in the target frame image is:
[0023]
[0024] in, Indicates The moving distance of the target area in the target frame image, Indicates The lateral movement distance of the target area in the target frame image, Indicates The moving characteristic factor of the target area in the target frame image, Indicates The angle change of the target frame image, Indicates The target frame image corresponds to the focal length of the camera.
[0025] Optionally, the obtaining of the overall displacement of the target area in the target frame sequence includes the following specific methods:
[0026] For the target area, obtain the target frame image corresponding to the first appearance of the target area in the target frame sequence, and the target frame image corresponding to the last appearance, which are recorded as the initial target frame image and the final target frame image of the target area, respectively; according to the difference between the real-time rotation parameters of the camera of the initial target frame image and the final target frame image, the circumscribed rectangle and the center of mass position of the target area, and the focal length of the camera corresponding to the initial target frame image, recalculate the moving distance and use it as the overall displacement of the target area in the target frame sequence.
[0027] Optionally, the obtaining of the behavior deviation factor of the target area in each target frame image includes the following specific methods:
[0028] According to the change of the centroid position of the target area in the adjacent target frame images, combined with the moving characteristic factor of the target area in the target frame image, the center change factor of the target area in each target frame image is obtained;
[0029] Based on the grayscale change of the target area in the adjacent target frame images and the moving characteristic factor of the target area in the target frame images, the overlapping change degree of the target area in each target frame image is obtained;
[0030] No. The calculation method of the action deviation factor of the target area in the target frame image is:
[0031]
[0032] in, Indicates The action deviation factor of the target area in the target frame image, Indicates The center change factor of the target area in the target frame image, Indicates The degree of overlap change of the target area in the target frame image, Represents the mean value of the overlapping variation of the target area in different target frame images. represents the absolute value function.
[0033] Optionally, the obtaining of the center change factor of the target area in each target frame image includes the following specific methods:
[0034]
[0035] in, Indicates The center change factor of the target area in the target frame image, Indicates The moving characteristic factor of the target area in the target frame image, and Respectively represent The target frame image and the The length of the bounding rectangle of the target area in the target frame image, and Respectively represent The target frame image and the The width of the bounding rectangle of the target area in the target frame image. and Respectively represent The target frame image and the The distance between the centroid of the target area in the target frame image and the left side of the circumscribed rectangle, and Respectively represent The target frame image and the The distance between the centroid of the target area in the target frame image and the upper side of the circumscribed rectangle.
[0036] Optionally, the obtaining of the overlapping change degree of the target area in each target frame image includes the following specific methods:
[0037] For The target frame image and the The target frame image is obtained by adjusting the rectangular area framed by the circumscribed rectangle of the target area in the two target frame images to the same size based on the moving characteristic factor of the target area; the two rectangular areas of the same size after adjustment are differentiated, and the obtained rectangular difference area is recorded as the target area in the first The difference area of the target frame image;
[0038] Pass the target area in The centroid of the corresponding area in the target frame image is located in the differential area, and parallel lines of the horizontal and vertical sides of the circumscribed rectangle are drawn to divide the differential area into four partitions. The calculation method of the overlapping change degree of the target area in the target frame image is:
[0039]
[0040] in, Indicates The degree of overlap change of the target area in the target frame image, Indicates that the target area is The first The number of pixels belonging to the target area in the partition, Indicates that the target area is The first The area of the partition, Indicates that the target area is The first The mean of the difference values of all pixels in a partition; Represents the normalization function.
[0041] Optionally, the obtaining of the abnormal behavior indicator of the target area includes the following specific methods:
[0042] The accumulated value of the moving distance of the target area in all the target frame images in which the target area exists is recorded as the moving distance of the target area in the target frame sequence;
[0043] For the target area, the target frame image containing the target area in the target frame sequence and having a moving distance in the target area is obtained, which is recorded as the effective target frame image of the target area. The calculation method of the abnormal behavior index of the target area is:
[0044]
[0045] in, Indicators of abnormal behavior in the target area, Indicates the number of valid target frame images of the target area, Indicates the target area The monitoring rotation factor of the effective target frame image, Indicates the target area The behavior deviation factor of the target area in the effective target frame image, represents the overall displacement of the target area in the target frame sequence, Indicates the moving distance of the target area in the target frame sequence.
[0046] The present invention also proposes an optimized transmission system for security monitoring videos based on the Internet of Things, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the above method are implemented.
[0047] The beneficial effects of the present invention are as follows: the present invention extracts the target frame image and analyzes the camera angle change of the target frame image, so as to preliminarily reflect the distortion of each target frame image and provide a basis for the subsequent analysis of the target area in the target frame image; the moving distance of the target area in the adjacent target frame image is obtained, which comprehensively considers the position change of the target area in the image, the change of the distance between the target and the camera, and the position change caused by the rotation of the camera, so as to avoid quantifying the moving distance based only on the position change in the image, resulting in inaccurate position change analysis and behavior analysis of the target, thereby affecting the subsequent displacement analysis of the target area; by analyzing the moving distance and the overall displacement of the target area, the abnormal wandering behavior of the target area can be reflected through the difference between the moving distance and the overall displacement. And participate in the quantification of abnormal behavior indicators; then, through the relative change of the centroid position of the target area in the circumscribed rectangle in the adjacent target frame images, and through the differential analysis of grayscale overlap changes, the behavior deviation factor is quantified to reflect the change of the target behavior morphology, and the abnormal behavior is analyzed from two aspects: the change of the target area itself and the movement change; by analyzing the movement change relationship of the target in the security monitoring video and the morphological change of the target itself, the abnormal behavior indicators of the target area are quantified, and the target frame images are screened, so that the corresponding video frame images of the targets with abnormal behavior can be transmitted with fidelity, and the targets with normal behavior only transmit the target area with fidelity, while the non-target frame images are filtered and then compressed for transmission, thereby effectively improving the compression rate of the security monitoring video, and then improving the transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0049] Figure 1 A schematic flow chart of a security surveillance video optimization transmission method based on the Internet of Things provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0051] See also Figure 1, which shows a flow chart of a security monitoring video optimization transmission method based on the Internet of Things provided by an embodiment of the present invention, the method comprising the following steps:
[0052] Step S001: Obtain security monitoring video, and record the real-time rotation parameters of the camera in each video frame through the pan / tilt system.
[0053] The purpose of this embodiment is to transmit security monitoring videos through the Internet of Things and optimize the transmission process. Therefore, it is necessary to collect security monitoring videos first. At the same time, the camera of the security monitoring system is arranged on the pan-tilt head, and the hemispherical angle coverage is achieved through multi-angle rotation, so as to realize functions such as target tracking and trajectory prediction. At the same time, it is necessary to record the real-time rotation parameters of the camera under the pan-tilt head corresponding to each video frame image in the security monitoring video.
[0054] Specifically, the security monitoring video of the current day is obtained through the camera in the security monitoring system. The security monitoring video contains several video frame images. The real-time rotation parameters corresponding to the camera under each video frame image are recorded by the pan / tilt. The real-time rotation parameters include the pitch angle and the steering angle of the camera under each video frame image, wherein the pitch angle ranges from the vertical , the steering angle range is lateral .
[0055] At this point, the security monitoring video has been collected, and several video frame images and the real-time rotation parameters of the camera under each video frame image have been obtained.
[0056] It should be noted that the video transmission of the security monitoring system has low transmission efficiency due to the large storage space it occupies, and the security monitoring system is mainly used to identify and track abnormal targets. Therefore, a large number of video frame images that only contain backgrounds in the security monitoring video need to be properly filtered to reduce the storage space they occupy, thereby improving transmission efficiency; and for video frame images where targets appear, it is necessary to further analyze whether the targets have abnormal behavior. For normal targets, only the targets need to be guaranteed to be undistorted, and the rest of the background can still be filtered; and for targets with abnormal behavior, it is necessary to ensure that the entire video frame image is not distorted to ensure the identification and extraction of abnormal behavior; thereby reducing the storage space occupied by security monitoring videos through filtering in different ranges and degrees, thereby achieving optimized transmission of security monitoring videos.
[0057] It should be further explained that, for continuous frame images with target areas, since the camera in the security monitoring system has a target tracking function, it will automatically adjust the camera's monitoring direction as the target moves. Therefore, it is necessary to analyze the camera's monitoring direction corresponding to the continuous frame images, quantify the monitoring rotation factor used to reflect the camera's rotation, and analyze the rotation angle of the camera in adjacent video frame images through the monitoring rotation factor to avoid image distortion caused by excessive rotation angles and affect target tracking and abnormal behavior judgment; and when the camera is not moving or the rotation amplitude is small, the target may still be moving, that is, it is far away from the camera or approaching the camera. It is necessary to analyze and quantify its movement characteristics, and then quantify the target's movement distance in the continuous frame images as a correction parameter for the movement position; then, by analyzing the difference in movement distance and the relationship between the movement distance and the overall displacement, combined with the morphological changes of the target area itself in adjacent frame images, the abnormal behavior is analyzed and judged.
[0058] Step S002: extract a number of target frame images from the security monitoring video to obtain a number of target frame sequences; obtain a monitoring rotation factor of each target frame image according to the real-time rotation parameter changes of the camera under adjacent target frame images in the target frame sequence.
[0059] Preferably, in one embodiment of the present invention, a plurality of target frame images are extracted from a security monitoring video to obtain a plurality of target frame sequences, and the specific method includes:
[0060] A training data set is formed by using several video frame images in a security surveillance video, and the target area in each video frame image is extracted through a semantic segmentation network. The background area in the video frame image is manually marked as 0 and the target area is marked as 1. The loss function adopts a cross entropy loss function to train the semantic segmentation network; then each video frame image is input into the trained semantic segmentation network, and the target area in each video frame image is output; the video frame image with the target area is recorded as the target frame image, and the video frame image without the target area is recorded as the non-target frame image; wherein the construction and training process of the semantic segmentation network are prior arts, and will not be repeated in this embodiment.
[0061] Furthermore, continuous target frame images are formed into a sequence, recorded as a target frame sequence, and a plurality of target frame sequences are obtained.
[0062] It should be noted that since there is a target area in the target frame image, the camera will track the target, thereby driving the camera to rotate, while since there is no target area in the non-target frame image, the camera is fixed. Therefore, it is necessary to first obtain the target frame sequence composed of the target frame images to monitor the analysis of the rotation factor.
[0063] Preferably, in one embodiment of the present invention, the monitoring rotation factor of each target frame image is obtained according to the real-time rotation parameter change of the camera under adjacent target frame images in the target frame sequence, and the specific method includes:
[0064] For any target frame image, according to the real-time rotation parameters of the camera under the target frame image and the adjacent previous target frame image, the angle change of the target frame image is obtained through the pitch angle change and the steering angle change. The angle change is the angle between the orientation of the camera of the target frame image and the orientation of the camera of the adjacent previous target frame image. The ratio of is recorded as the angle change factor of the target frame image; wherein the angle can be obtained by changing the pitch angle and the steering angle by using the existing method, which will not be described in detail in this embodiment, and the angle change amount and The ratio of is used to normalize the angle change to obtain the angle change factor.
[0065] Furthermore, a reference number is preset. In this embodiment, the reference number is described as 10. The target frame images before and after the target frame image are used as reference frame images of the target frame image. It is particularly noted that if the target frame image is located at or close to the boundary of the target frame sequence so that a sufficient number of reference frame images cannot be obtained, the actual target frame image is used as the reference frame image; then The calculation method of the monitoring rotation factor of the target frame image is:
[0066]
[0067] in, Indicates The monitoring rotation factor of the target frame image, Indicates The number of reference frame images of the target frame image, Indicates The angle change factor of the target frame image, Indicates The target frame image Angular variation factor of the reference frame image.
[0068] It should be noted that the angle change factors between consecutive target frame images should be equivalent. If there is a large difference in the angle change factor between the target frame image and other target frame images around it, the angle change difference is large, and there is abnormal rotation that may cause image distortion, and the corresponding monitoring rotation factor is larger.
[0069] At this point, the target frame images are extracted and the camera angle changes of the target frame images are analyzed to preliminarily reflect the distortion of each target frame image and provide a basis for the subsequent analysis of the target area in the target frame image.
[0070] Step S003, according to the change of the area ratio of the target area in the adjacent target frame images, obtain the moving characteristic factor of the target area in each target frame image; based on the position change of the target area in the adjacent target frame images, adjust through the moving characteristic factor, combined with the real-time rotation parameter change of the camera, to obtain the moving distance of the target area in each target frame image.
[0071] It should be noted that if the target is far away from the camera or approaches the camera, the camera will not rotate or rotate slightly. Therefore, it is necessary to further analyze the changes in the target area and image proportion in consecutive target frame images. Changes in the area proportion indicate that it is approaching or moving away from the camera. The camera is stationary or rotates slightly but the target is still moving. This is used as the target's movement feature to correct the position change of the target in consecutive target frame images and quantify the target's movement distance in adjacent target frame images.
[0072] Preferably, in one embodiment of the present invention, the moving characteristic factor of the target area in each target frame image is obtained according to the change in the area ratio of the target area in the image in adjacent target frame images, and the specific method includes:
[0073] For any target frame sequence, the optical flow method is used to match the target area in each target frame image, and the target frame sequence contains several target areas, and each target area exists in several target frame images; unless otherwise specified, this embodiment and the subsequent processing take any one of the target areas as an example, and the target areas mentioned subsequently are all the target areas, and the target frame sequences mentioned subsequently are all the target frame sequences corresponding to the target area.
[0074] Furthermore, a bounding rectangle is obtained for the corresponding area of the target area in each target frame image, wherein the horizontal side of the bounding rectangle is parallel to the horizontal side of the image, and the vertical side is parallel to the vertical side of the image, and is recorded as the bounding rectangle of the target area in each target frame image. Then The calculation method of the moving characteristic factor of the target area in the target frame image is:
[0075]
[0076] in, Indicates The moving characteristic factor of the target area in the target frame image, Indicates the length of the video frame image (horizontal side), and Respectively represent The target frame image and the The length of the bounding rectangle of the target area in the target frame image (horizontal side), Indicates the width (vertical side) of the video frame image. and Respectively represent The target frame image and the The width (vertical side) of the bounding rectangle of the target area in the target frame image.
[0077] It should be noted that if the target area in the adjacent target frame image is close to or far away from the camera, it will cause the length and width of the circumscribed rectangle of the target area to increase or decrease at the same time. The difference in the length and width of the circumscribed rectangle of the target area in the adjacent target frame image is used to quantify the movement characteristic factor of the target area in the previous target frame image; it should be noted in particular that if the corresponding target area does not exist in the next video frame image (target frame image), the movement characteristic factor of the target area will not be calculated for the current target frame image, and the subsequent moving distance will not be calculated.
[0078] Preferably, in one embodiment of the present invention, based on the position change of the target area in adjacent target frame images, the moving characteristic factor is adjusted, and combined with the real-time rotation parameter change of the camera, the moving distance of the target area in each target frame image is obtained, including the specific method of:
[0079] The centroid of the corresponding area of the target area in each target frame image is obtained, and a coordinate system is constructed with the lower left corner of each target frame image as the coordinate origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical upward direction as the positive direction of the vertical axis to obtain the coordinates of each centroid; the Euclidean distance between the centroids of the corresponding areas of the target area in adjacent target frame images is used as the lateral movement distance of the target area in the previous target frame image in the adjacent target frame image.
[0080] It should be noted that the Euclidean distance between the centroids can only reflect the moving distance of the target in the image. If the target moves toward or away from the camera, the centroid position usually does not change. At this time, it is necessary to adjust the lateral moving distance in combination with the moving characteristic factor. The length and width of the circumscribed rectangle change in the same direction, that is, they increase or decrease at the same time, and the increase or decrease has no effect on the correction of the lateral moving distance. The consistent change direction makes the moving characteristic factor usually positive. If the moving characteristic factor is 0, the distance between the target and the camera has not changed and no correction is required. If the moving characteristic factor is a negative value, the target area has undergone special changes, and the moving characteristic factor is no longer applicable to the correction of the lateral moving distance. The moving distance also needs to consider the change in the camera angle. The moving distance of the target area is quantified by the rotation angle between adjacent target frame images and the focal length of the camera.
[0081] Specifically, The calculation method of the moving distance of the target area in the target frame image is:
[0082]
[0083] in, Indicates The moving distance of the target area in the target frame image, Indicates The lateral movement distance of the target area in the target frame image, Indicates The moving characteristic factor of the target area in the target frame image, Indicates The angle change of the target frame image, Indicates The target frame image corresponds to the focal length of the camera (obtained directly from the camera).
[0084] It should be noted that the lateral movement distance is corrected by moving the characteristic factor. The larger the moving characteristic factor, the more the lateral movement distance needs to be increased. The angle change is calculated by the current target frame image and the adjacent previous target frame image, and the moving distance is to quantify the change between the target area in the current target frame image and the adjacent subsequent target frame image. Therefore, the angle change of the subsequent target frame image is used for calculation, and the focal length of the camera corresponding to the current target frame image is combined to quantify the distance presented in the image by the angle change, so as to obtain the moving distance of the target area through the Euclidean norm.
[0085] At this point, the moving distance of the target area in the adjacent target frame images is obtained, which comprehensively considers the position change of the target area in the image, the change in the distance between the target and the camera, and the position change caused by the rotation of the camera, so as to avoid quantifying the moving distance based only on the position change in the image, resulting in inaccurate position change analysis and behavior analysis of the target, thereby affecting the subsequent displacement analysis of the target area.
[0086] Step S004: according to the target area in each target frame image in the target frame sequence and its moving distance, the overall displacement of the target area in the target frame sequence is obtained; according to the change of the center of mass position and the grayscale change of the target area in the adjacent target frame images, combined with the moving characteristic factor of the target area in the target frame image, the behavior deviation factor of the target area in each target frame image is obtained; combined with the monitoring rotation factor of each target frame image, the abnormal behavior index of the target area is obtained by comparing the difference between the overall displacement of the target area in the target frame sequence and the moving distance of the target area in each target frame image.
[0087] It should be noted that for targets with abnormal behavior, their morphology will change between adjacent frames, that is, there will be movement behavior with morphological changes. At the same time, the moving distance of normal passers-by should be the same as the overall displacement, that is, they only pass by the camera. However, under abnormal behavior, they will move multiple times within the camera's monitoring range and there will be wandering behavior, resulting in a difference between the overall moving distance and the overall displacement, so as to obtain the abnormal behavior index.
[0088] It should be further explained that for the morphological changes of the target area, that is, even if the scales of the target areas between adjacent target frame images are equal, the specific morphologies are quite different, resulting in a smaller overlap. Based on the difference of the target area and the analysis of the position change of the target area centroid in the circumscribed rectangular frame, the morphological changes caused by the abnormal behavior of the target morphology are quantified; and the wandering behavior under abnormal behavior is analyzed by the moving distance of the target area between adjacent target frame images. The distance traveled by a normal passerby under the camera is the same as the displacement, and there will be no difference; and the wandering behavior is due to the reciprocating movement of the target within the camera range, resulting in the distance accumulated by the moving distance, which is quite different from the displacement quantified by the first and last target frames within the frame segment range, and then this is used as the quantitative feature of the abnormal behavior indicator.
[0089] Preferably, in one embodiment of the present invention, the overall displacement of the target area in the target frame sequence is obtained according to the target area in each target frame image in the target frame sequence and its moving distance, and the specific method includes:
[0090] For the target area, the target frame image corresponding to the first appearance of the target area in the target frame sequence and the target frame image corresponding to the last appearance are obtained, which are recorded as the initial target frame image and the final target frame image of the target area, respectively; based on the difference between the real-time rotation parameters of the camera of the initial target frame image and the final target frame image, the circumscribed rectangle and the center of mass position of the target area, and the focal length of the camera corresponding to the initial target frame image, the moving distance is recalculated according to the calculation method of the moving distance of the target area in each target frame image, and is used as the overall displacement of the target area in the target frame sequence.
[0091] Furthermore, the accumulated value of the moving distance of the target area in all target frame images in which the target area exists is recorded as the moving distance of the target area in the target frame sequence.
[0092] It should be noted that the moving distance is obtained by accumulating the moving distance of the target area, and the overall displacement is quantified according to the change of the target area in the first frame and the last frame. The moving distance is then compared with the overall displacement to determine whether the target has wandering behavior.
[0093] Preferably, in one embodiment of the present invention, the behavior deviation factor of the target area in each target frame image is obtained according to the change of the centroid position and the grayscale change of the target area in the adjacent target frame images, combined with the movement characteristic factor of the target area in the target frame image, and the specific method includes:
[0094] According to the centroid of the corresponding area of the target area in each target frame image and the circumscribed rectangle of the target area in each target frame image, the first The center change factor of the target area in the target frame image , which is calculated as:
[0095]
[0096] in, Indicates The center change factor of the target area in the target frame image, Indicates The moving characteristic factor of the target area in the target frame image, and Respectively represent The target frame image and the The length of the bounding rectangle of the target area in the target frame image, and Respectively represent The target frame image and the The width of the bounding rectangle of the target area in the target frame image. and Respectively represent The target frame image and the The distance between the centroid of the target area in the target frame image and the left side of the circumscribed rectangle, and Respectively represent The target frame image and the The distance between the centroid of the target area in the target frame image and the upper side of the circumscribed rectangle.
[0097] It should be noted that, by analyzing the relative changes in the center of mass position in adjacent target frame images, the greater the relative change within the circumscribed rectangle, the greater the change in the morphology of the target area. Since the center of mass is obtained based on the grayscale distribution of all pixels in the target area, the morphological changes can be reflected to a certain extent by the change in the center of mass. At the same time, in order to avoid the influence of size changes, corrections are made based on the moving characteristic factor. The smaller the moving characteristic factor, the smaller its influence on the center of mass, and the larger the corresponding center change factor.
[0098] Further, the The target frame image and the The target frame image is taken as an example, and the rectangular area framed by the circumscribed rectangle of the target area in the two target frame images is adjusted to the same size based on the moving characteristic factor of the target area. The rectangular area in the target frame image is adjusted to the first The size of the rectangular area of the target frame image is adjusted by interpolation if it is increased, and downsampling if it is decreased. The specific process is the prior art and will not be described in detail in this embodiment. The two rectangular areas of the same size after adjustment are differentiated. The difference method adopts the absolute value difference, that is, the absolute value of the difference is obtained for the pixels at the same position, and the obtained rectangular difference area is recorded as the target area. The difference area of the target frame image; the target area is The centroid of the corresponding area in the target frame image is located in the differential area. Lines parallel to the horizontal and vertical sides of the circumscribed rectangle are drawn to divide the differential area into four partitions. The calculation method of the overlapping change degree of the target area in the target frame image is:
[0099]
[0100] in, Indicates The degree of overlap change of the target area in the target frame image, Indicates that the target area is The first The number of pixels belonging to the target area in the partition, Indicates that the target area is The first The area of the partition, Indicates that the target area is The first The mean of the difference values of all pixels in a partition, that is, the mean of the difference values; Represents the weight normalization function, and the normalized object is the four partitions in the difference area. .
[0101] It should be noted that by analyzing the mean of the differential values in each partition and taking the proportion of the number of pixels in the target area in different partitions as the weight, the degree of overlapping change is quantified according to the size of the differential value; subsequently, the mean of the overlapping change degree is obtained, and the overlapping change degree is adjusted by the center change factor. The larger the center change factor, the less referenceable the corresponding overlapping change degree is. The difference between the overlapping change degree and its mean is used as the action deviation factor of the target area in the corresponding target frame image.
[0102] Specifically, The calculation method of the action deviation factor of the target area in the target frame image is:
[0103]
[0104] in, Indicates The action deviation factor of the target area in the target frame image, Indicates The center change factor of the target area in the target frame image, Indicates The degree of overlap change of the target area in the target frame image, Represents the mean value of the overlapping variation of the target area in different target frame images. represents the absolute value function.
[0105] Preferably, in one embodiment of the present invention, in combination with the monitoring rotation factor of each target frame image, by comparing the difference between the overall displacement of the target area in the target frame sequence and the moving distance of the target area in each target frame image, the abnormal behavior index of the target area is obtained, including the specific method of:
[0106] For the target area, the target frame image containing the target area in the target frame sequence and having a moving distance in the target area is obtained, which is recorded as the effective target frame image of the target area. The calculation method of the abnormal behavior index of the target area is:
[0107]
[0108] in, Indicators of abnormal behavior in the target area, Indicates the number of valid target frame images of the target area, Indicates the target area The monitoring rotation factor of the effective target frame image, Indicates the target area The behavior deviation factor of the target area in the effective target frame image, represents the overall displacement of the target area in the target frame sequence, Indicates the moving distance of the target area in the target frame sequence.
[0109] It should be noted that the behavior deviation factor is quantified based on the change in the target area between the current target frame image and the adjacent subsequent target frame image, and the monitoring rotation factor of the current target frame image is introduced for adjustment. The larger the monitoring rotation factor, the greater the possibility that the camera corresponding to the current target frame image rotates abnormally, the more likely the image is distorted, and the lower the credibility of the corresponding behavior deviation factor. The larger the behavior deviation factor, the greater the possibility of abnormal behavior and the larger the abnormal behavior index. By comparing the moving distance with the overall displacement, the closer the overall displacement is to the moving distance, the smaller the possibility of the target moving back and forth within the camera range, the smaller the possibility of wandering behavior, the smaller the possibility of abnormal behavior and the smaller the abnormal behavior index.
[0110] So far, by analyzing the moving distance and overall displacement of the target area, the difference between the moving distance and the overall displacement is used to reflect the abnormal wandering behavior of the target area and participate in the quantification of abnormal behavior indicators; then, through the relative change of the centroid position of the target area in the circumscribed rectangle in adjacent target frame images, and through differential analysis of grayscale overlap changes, the behavior deviation factor is quantified to reflect the change in the target behavior morphology, and the abnormal behavior is analyzed from two aspects: the change of the target area itself and the change of movement.
[0111] Step S005: Filter the target frame images according to the abnormal behavior indicators of the target area, and compress and transmit them in combination with the non-target frame images.
[0112] Specifically, the abnormal behavior indicators are obtained for all target areas in all target frame sequences according to the above method. It should be noted that the target area here is the matched target area obtained by the optical flow method. The same target area in different target frame images after the optical flow method is one target area here, and the target areas matched as the same target area are recorded as one target area; the abnormal behavior indicators of all target areas are linearly normalized, and the results are used as the abnormal behavior factors of each target area.
[0113] Furthermore, an abnormal behavior threshold is preset. In this embodiment, the abnormal behavior threshold is described as 0.7. For any target area, if the abnormal behavior factor of the target area is greater than or equal to the abnormal behavior threshold, the target frame image containing the target area is recorded as the first target frame image; if the abnormal behavior factor of the target area is less than the abnormal behavior threshold, the target frame image containing the target area and not the first target frame image is recorded as the second target frame image, and mean filtering is performed on all pixels in the background area of the second target frame image to obtain a filtered second target frame image; mean filtering is performed on all pixels in the non-target frame image to obtain a filtered non-target frame image.
[0114] Furthermore, for the first target frame image and the filtered second target frame image, Huffman coding lossless compression method is used for compression transmission, and for the filtered non-target frame image, predictive coding lossy compression method is used for compression transmission, thereby achieving optimized transmission of security monitoring video.
[0115] At this point, by analyzing the movement change relationship of the target in the security surveillance video and the morphological changes of the target itself, the abnormal behavior indicators of the target area are quantified, and the target frame images are screened, so that the corresponding video frame images of the target with abnormal behavior can be transmitted with fidelity, and the target with normal behavior only transmits the target area with fidelity, while the non-target frame images are filtered and then compressed for transmission, thereby effectively improving the compression rate of the security surveillance video and thus improving the transmission efficiency.
[0116] Another embodiment of the present invention provides a security monitoring video optimization transmission system based on the Internet of Things, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the above-mentioned method steps S001 to S005 are implemented.
[0117] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A security monitoring video optimization transmission method based on the Internet of Things, characterized in that: The method comprises the following steps: Obtain security surveillance video and use the PTZ to record the real-time rotation parameters of the camera in each video frame; Extracting a number of target frame images from the security monitoring video to obtain a number of target frame sequences; obtaining a monitoring rotation factor for each target frame image according to the real-time rotation parameter changes of the camera under adjacent target frame images in the target frame sequence; The specific method of obtaining the monitoring rotation factor of each target frame image includes: For any target frame image, according to the real-time rotation parameters of the camera under the target frame image and the adjacent previous target frame image, the angle change of the target frame image is obtained through the pitch angle change and the steering angle change; the angle change of the target frame image is compared with The ratio of is recorded as the angle change factor of the target frame image; The reference number of target frame images before and after the target frame image are used as the reference frame images of the target frame image. The calculation method of the monitoring rotation factor of the target frame image is: in, Indicates The monitoring rotation factor of the target frame image, Indicates The number of reference frame images of the target frame image, Indicates The angle change factor of the target frame image, Indicates The target frame image Angle variation factor of the reference frame image; According to the change of the area ratio of the target area in the adjacent target frame images, the moving characteristic factor of the target area in each target frame image is obtained; based on the position change of the target area in the adjacent target frame images, the moving characteristic factor is adjusted, and combined with the real-time rotation parameter change of the camera, the moving distance of the target area in each target frame image is obtained; The specific method for obtaining the moving characteristic factor of the target area in each target frame image is as follows: Obtain the bounding rectangle of the corresponding area of the target area in each target frame image, recorded as the bounding rectangle of the target area in each target frame image. The calculation method of the moving characteristic factor of the target area in the target frame image is: in, Indicates The moving characteristic factor of the target area in the target frame image, Indicates the length of the video frame image. and Respectively represent The target frame image and the The length of the bounding rectangle of the target area in the target frame image, Indicates the width of the video frame image. and Respectively represent The target frame image and the The width of the bounding rectangle of the target area in the target frame image; According to the target area in each target frame image in the target frame sequence and its moving distance, the overall displacement of the target area in the target frame sequence is obtained; according to the centroid position change and grayscale change of the target area in the adjacent target frame images, combined with the moving characteristic factor of the target area in the target frame image, the behavior deviation factor of the target area in each target frame image is obtained; combined with the monitoring rotation factor of each target frame image, the abnormal behavior index of the target area is obtained by comparing the difference between the overall displacement of the target area in the target frame sequence and the moving distance of the target area in each target frame image; The specific method of obtaining the behavior deviation factor of the target area in each target frame image includes: According to the change of the centroid position of the target area in the adjacent target frame images, combined with the moving characteristic factor of the target area in the target frame image, the center change factor of the target area in each target frame image is obtained; Based on the grayscale change of the target area in the adjacent target frame images and the moving characteristic factor of the target area in the target frame images, the overlapping change degree of the target area in each target frame image is obtained; No. The calculation method of the behavior deviation factor of the target area in the target frame image is: in, Indicates The behavior deviation factor of the target area in the target frame image, Indicates The center change factor of the target area in the target frame image, Indicates The degree of overlap change of the target area in the target frame image, Represents the mean value of the overlapping variation of the target area in different target frame images. represents the absolute value function; According to the abnormal behavior indicators of the target area, the target frame images are screened and combined with the non-target frame images for compression and transmission.
2. The method for optimizing transmission of security surveillance video based on the Internet of Things according to claim 1 is characterized in that: The specific method of obtaining the moving distance of the target area in each target frame image includes: Obtain the centroid of the corresponding area of the target area in each target frame image, construct a coordinate system with the lower left corner of each target frame image as the coordinate origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical upward direction as the positive direction of the vertical axis, and obtain the coordinates of each centroid; take the Euclidean distance between the centroids of the corresponding areas of the target area in adjacent target frame images as the lateral movement distance of the target area in the previous target frame image in the adjacent target frame image; No. The calculation method of the moving distance of the target area in the target frame image is: in, Indicates The moving distance of the target area in the target frame image, Indicates The lateral movement distance of the target area in the target frame image, Indicates The moving characteristic factor of the target area in the target frame image, Indicates The angle change of the target frame image, Indicates The target frame image corresponds to the focal length of the camera.
3. The method for optimizing transmission of security surveillance video based on the Internet of Things according to claim 2 is characterized in that: The specific method of obtaining the overall displacement of the target area in the target frame sequence includes: For the target area, obtain the target frame image corresponding to the first appearance of the target area in the target frame sequence, and the target frame image corresponding to the last appearance, which are recorded as the initial target frame image and the final target frame image of the target area, respectively; according to the difference between the real-time rotation parameters of the camera of the initial target frame image and the final target frame image, the circumscribed rectangle and the center of mass position of the target area, and the focal length of the camera corresponding to the initial target frame image, recalculate the moving distance and use it as the overall displacement of the target area in the target frame sequence.
4. The method for optimizing transmission of security surveillance video based on the Internet of Things according to claim 1 is characterized in that: The specific method of obtaining the center change factor of the target area in each target frame image includes: in, Indicates The center change factor of the target area in the target frame image, Indicates The moving characteristic factor of the target area in the target frame image, and Respectively represent The target frame image and the The length of the bounding rectangle of the target area in the target frame image, and Respectively represent The target frame image and the The width of the bounding rectangle of the target area in the target frame image. and Respectively represent The target frame image and the The distance between the centroid of the target area in the target frame image and the left side of the circumscribed rectangle, and Respectively represent The target frame image and the The distance between the centroid of the target area in the target frame image and the upper side of the circumscribed rectangle.
5. The method for optimizing transmission of security surveillance video based on the Internet of Things according to claim 2 is characterized in that: The specific method of obtaining the overlapping change degree of the target area in each target frame image includes: For The target frame image and the The target frame image is obtained by adjusting the rectangular area framed by the circumscribed rectangle of the target area in the two target frame images to the same size based on the moving characteristic factor of the target area; the two rectangular areas of the same size after adjustment are differentiated, and the obtained rectangular difference area is recorded as the target area in the first The difference area of the target frame image; Pass the target area in The centroid of the corresponding area in the target frame image is located in the differential area, and parallel lines of the horizontal and vertical sides of the circumscribed rectangle are drawn to divide the differential area into four partitions. The calculation method of the overlapping change degree of the target area in the target frame image is: in, Indicates The degree of overlap change of the target area in the target frame image, Indicates that the target area is The first The number of pixels belonging to the target area in the partition, Indicates that the target area is The first The area of the partition, Indicates that the target area is The first The mean of the difference values of all pixels in a partition; Represents the normalization function.
6. The method for optimizing transmission of security surveillance video based on the Internet of Things according to claim 1 is characterized in that: The specific method of obtaining the abnormal behavior indicator of the target area includes: The accumulated value of the moving distance of the target area in all the target frame images in which the target area exists is recorded as the moving distance of the target area in the target frame sequence; For the target area, the target frame image containing the target area in the target frame sequence and having a moving distance in the target area is obtained, which is recorded as the effective target frame image of the target area. The calculation method of the abnormal behavior index of the target area is: in, Indicators of abnormal behavior in the target area, Indicates the number of valid target frame images of the target area, Indicates the target area The monitoring rotation factor of the effective target frame image, Indicates the target area The behavior deviation factor of the target area in the effective target frame image, represents the overall displacement of the target area in the target frame sequence, Indicates the moving distance of the target area in the target frame sequence.
7. A security surveillance video optimization transmission system based on the Internet of Things, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the security monitoring video optimization transmission method based on the Internet of Things as described in any one of claims 1-6 are implemented.
