Target tracking and positioning system based on video analysis

Abstract

The invention relates to the field of video monitoring, in particular to a target tracking and positioning system based on video analysis. The system comprises a data detection module, a feature generation module, a calculation module and a tracking and positioning module, and is used for obtaining a human body region feature map of an adjacent frame image in real time based on a preset human body classification model, calculating portrait feature data based on the human body region feature map, and performing grid division on the human body region feature map based on the portrait feature data, the method comprises the steps of obtaining a human body grid feature map, calculating the contribution degree of each grid in the human body grid feature map to human body classification so as to construct contribution degree sequences, calculating the distance between the contribution degree sequences, and adjusting the angle of a camera based on the distance so as to realize tracking and positioning of a target. Through the technical scheme of the invention, the camera can be controlled to change the angle along with the movement of a person, so that the person has a proper screen-to-body ratio, and the action information of the person can be completely monitored.

Description

Technical Field

[0001] The present invention generally relates to the field of video surveillance and more specifically to a target tracking and positioning system based on video analysis. Background Art

[0002] Reasonable camera shooting angle can obtain better shooting effect and visual experience. By adjusting the camera angle, users can get better visual experience, clearer images and more accurate information; for video surveillance systems, suitable camera angles can increase the monitoring range and field of view, improve monitoring effect and safety.

[0003] In the related technology, for example, the patent with application publication number CN118071826A discloses a target positioning method and system based on video surveillance, including obtaining video stream data, establishing multiple target detection frames, using each target detection frame to extract multi-channel gradient features and multi-channel color features for different target areas of the video image, and generating multiple feature sub-maps; splicing the feature sub-maps according to the intersection and union ratio between any two target detection frames, calculating the motion blur kernel of the spliced ​​target feature map to calculate the blur, and adjusting the motion blur kernel compensation until the blur reaches a preset value, performing a deblurring operation on the target feature map, and combining with the correlation filter to calculate the position coordinates of the target in the continuous frame image to complete the target tracking and positioning.

[0004] In the prior art, the camera generally needs to maintain a certain distance from the person in order to achieve a suitable screen-to-body ratio. After that, the position and range of the person are calibrated, and then the angle that needs to be adjusted is calculated based on the recognition of several actions of the person, and then the angle of the camera is manually adjusted to achieve a better monitoring effect.

[0005] This solution can control the camera to adjust the angle according to the direction of the person's movement when someone is moving, so that the camera always adjusts the angle to follow the changes of the person, solving the problem that the camera cannot perform monitoring in a timely and accurate manner due to different movement speeds and different movement methods of people. Summary of the invention

[0006] To solve one or more of the above technical problems, the present invention proposes a target tracking and positioning system based on video analysis. To this end, the present invention provides solutions in the following aspects.

[0007] The present invention provides a target tracking and positioning system based on video analysis, comprising: a data detection module, used for inputting video data shot by a camera in different scenes into a pre-trained person recognition network model for detection to obtain a human body region feature image set; a feature generation module, used for calculating the portrait feature data of each image in the human body region feature image set, and dividing the human body region feature map into grids based on the portrait feature data to obtain a human body grid feature map; a calculation module, used for calculating the contribution of each grid in the human body grid feature map to human body classification to construct a contribution sequence, wherein the human body grid feature map and the contribution sequence correspond one to one; and a tracking and positioning module, used for calculating the distance of the contribution sequence between adjacent images in the human body region feature image set, and adjusting the camera angle based on the distance to achieve tracking and positioning of the target.

[0008] By controlling the camera to change its angle as the person moves, the person can have a suitable screen-occupying ratio and the person's movement trajectory can be fully monitored. At the same time, gridding the image can more sensitively judge the person's movement information and enable the camera to quickly adjust to the appropriate angle, reducing angle shooting deviation and making the monitoring effect clearer and more accurate.

[0009] Preferably, the portrait feature data includes the ratio of the area of ​​the human body circumscribed matrix to the entire image area, the ratio of the human body area in the external matrix, the average human body density in rows and the average human body density in columns, including: The ratio of the area of ​​the human body circumscribed rectangle to the entire image area satisfies the relationship: ; The proportion of the human body area in the external matrix satisfies the relationship: ; The average density of pedestrians satisfies the relationship: ; The average density of the human body in the column satisfies the relationship: ; in, It represents the ratio of the area of ​​the human body's circumscribed rectangle to the entire image area. represents the rows of the image, represents the columns of the image, Indicates the proportion of the human body area within the circumscribed rectangle. represents the average density of pedestrians, represents the average density of human body, Represents the area of ​​the human body in the circumscribed rectangle, represents the area of ​​the circumscribed rectangle, Indicates the width of the bounding rectangle. Indicates the height of the bounding rectangle.

[0010] Preferably, the human body region feature map is divided into grids including the number of row grids and the number of column grids, including: The number of row grids satisfies the relationship: ; The number of column grids satisfies the relationship: ; in, Represents the number of row grids of the image, Represents the number of column grids of the image, It represents the ratio of the area of ​​the human body's circumscribed rectangle to the entire image area. Indicates the proportion of the human body area within the circumscribed rectangle. represents the average density of pedestrians, Represents the average density of human body.

[0011] In this way, gridding the image can more sensitively determine the person's action information, and adjust the camera to the appropriate angle, reducing angle shooting deviation, making the monitoring effect clearer and more accurate.

[0012] Preferably, the contribution of the grid human body classification satisfies the relationship: ; in, Indicates The contribution of each grid to human body classification. Indicates In the grid Line The pixel value at the column position.

[0013] Preferably, the distance adjustment of the camera angle includes: based on the contribution sequence, calculating the distance between two adjacent contribution sequences, comparing the distance with a preset threshold, if it is less than or equal to the preset threshold, outputting a first signal, and if it is greater than the preset threshold, outputting a second signal; in response to the first signal, the person in the image has not moved and there is no need to adjust the camera angle; in response to the second signal, the person in the image has moved and the camera angle needs to be adjusted.

[0014] Preferably, the method for adjusting the camera angle includes: based on the change in contribution of grids at the same position in the adjacent frame images, extracting the grids where the character area changes, and screening the grids where the contribution increases to obtain the direction of character movement; calculating the coordinates of the center point of the entire grid where the contribution increases, and controlling the camera to adjust to the center point.

[0015] The present invention has the following technical effects: 1. The camera can be controlled to change its angle as the character moves, so that the character has a suitable screen ratio and the character's movement trajectory can be fully monitored.

[0016] 2. Gridding the image can more sensitively determine the person's action information, and enable the camera to quickly adjust to the appropriate angle, reducing angle shooting deviation and making the monitoring effect clearer and more accurate.

[0017] 3. By calculating the average value of the grid divisions corresponding to all human body areas on the image, the overall monitoring image can be kept relatively stable, and the characters will not appear in different sizes, avoiding the characters being blocked. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 The present invention is a system block diagram of a target tracking and positioning system based on video analysis according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] It should be understood that when the terms "first", "second", etc. are used in the claims, descriptions, and drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the description and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0021] The present invention provides a target tracking and positioning system based on video analysis. Figure 1 As shown, a target tracking and positioning system based on video analysis includes modules 101 to 104, which are described in detail below.

[0022] The data detection module 101 is used to input the video data shot by the camera in different scenes into a pre-trained person recognition network model for detection to obtain a human body region feature image set.

[0023] Among them, videos shot by cameras in different environments are collected, including whether the lights are on, whether there are people in the room, etc. The videos are converted into images, and each image is given a classification label, which is divided into human body labels and no human body labels, to establish a human body classification data set.

[0024] In one embodiment, the application environment of the camera is simulated. The application environments of the camera include whether the lights are on, the lights are off, there are people in the house, or there are no people in the house. Videos are shot in different environments, converted into images, and an image is extracted every 10 frames. Each extracted image is given a label, which is divided into a human label and an unmanned label. A human classification data set is established. Videos are shot in different environments to enhance the diversity of images so that the trained model has stronger robustness.

[0025] Among them, the human classification model can use a convolutional neural network. The convolutional neural network can be trained using the human classification data set. After training, the new images taken by the camera are classified, and the CAM (feature visualization technology) algorithm is used to obtain the CAM grayscale image of the new image.

[0026] In one embodiment, a classification convolutional neural network is trained using a human classification data set, with an image as input and an output of 1 for human presence and 0 for human absence. The loss function of the model uses a cross entropy loss function, and training is stopped when the model reaches the maximum number of training times or the model loss is less than a preset loss threshold, and the optimal classification model is selected based on the accuracy of the model evaluation index. After completing the training of the classification convolutional neural network, a cam grayscale image of the new image is obtained using feature visualization technology. The larger the pixel value in the grayscale image, the greater the accuracy of the model in judging that there is a person in the image. The grayscale image is extracted for a connected domain to obtain the human body area in the image, which is the human body area feature map. In this solution, all the obtained human body area feature maps are used as a human body area feature atlas.

[0027] In this optional embodiment, portrait feature data is calculated based on a human body region feature map, wherein the portrait feature data includes the ratio of the human body circumscribed matrix area to the entire image area, the ratio of the human body region within the external matrix, the average human body density in rows, and the average human body density in columns.

[0028] In one embodiment, the circumscribed rectangle of each connected domain is determined, and portrait feature data is calculated. The portrait feature data includes the ratio of the area of ​​the human body circumscribed rectangle to the entire image area, the ratio of the human body area in each circumscribed rectangle, and the average human body density in rows and columns of the circumscribed rectangle. The number of pixel points in the connected domain of each circumscribed rectangle can be counted to obtain the area size S of each human body area. According to S, the height and width of the circumscribed rectangle, the ratio of the area of ​​the human body circumscribed rectangle to the entire image area, the ratio of the human body area in the circumscribed rectangle, the average human body density in rows and the average human body density in columns are obtained.

[0029] The ratio of the area of ​​the human body's circumscribed rectangle to the entire image area satisfies the relationship: ; The proportion of the human body area in the external matrix satisfies the relationship: ; The average density of pedestrians satisfies the relationship: ; The average density of human body satisfies the relationship: ; in, It represents the ratio of the area of ​​the human body's circumscribed rectangle to the entire image area. represents the rows of the image, represents the columns of the image, Indicates the proportion of the human body area within the circumscribed rectangle. represents the average density of pedestrians, represents the average density of human body, represents the area of ​​the human body in the circumscribed rectangle, represents the area of ​​the circumscribed rectangle, Indicates the width of the bounding rectangle. Indicates the height of the bounding rectangle.

[0030] Among them, when the image is divided into grids, if the grid is too large, the changes in the human body area will not be reflected in the grid. If the grid is too small, the requirement for calculation is large. The purpose of dividing the image into grids is to make the human body features in each grid more prominent. The human body features in the grid specifically refer to that the changes in the human body area in the grid are more sensitive.

[0031] Preferably, when the grid division is consistent with the size of the image, that is, one image is divided into one grid, the movement of the human body area in the grid does not change the area of ​​the human body area in the entire grid. Therefore, the image is grid-divided according to the ratio of the human body circumscribed rectangle area to the entire image area, the ratio of the human body area in the circumscribed rectangle, and the average density of the human body rows and columns. The smaller the ratio of the human body circumscribed rectangle area to the entire image area, the more grid divisions there are, the smaller the ratio of the human body area in the circumscribed rectangle, the more grid divisions there are, the smaller the row average density, the more grids there are in the image rows, the smaller the column average density, the more grids there are in the image columns, the ratio of the human body area in the circumscribed rectangle determines the number of grid divisions of the entire image, and the average density of the human body rows and columns respectively determines the number of grid divisions of the image rows and columns.

[0032] The human body region feature map is divided into grids including the number of row grids and the number of column grids, including: The number of row grids satisfies the relationship: ; The number of column grids satisfies the relationship: ; in, Represents the number of row grids of the image, Represents the number of column grids of the image, It represents the ratio of the area of ​​the human body's circumscribed rectangle to the entire image area. Indicates the proportion of the human body area within the circumscribed rectangle. represents the average density of pedestrians, Represents the average density of human body.

[0033] Preferably, the area of ​​the human body circumscribed rectangle accounts for 1 / 6 of the entire image area, the human body area within the circumscribed rectangle accounts for 1 / 2, and the average density of human body rows and columns are h / 2 and w / 2 respectively, then the image is divided into 24×24 grids.

[0034] Preferably, when there are multiple human body regions on an image, each human body region corresponds to a grid division, and the grid division of the entire image is an average value of each grid division.

[0035] The calculation module 103 is used to calculate the contribution of each grid in the human body grid feature map to the human body classification to construct a contribution sequence.

[0036] Among them, the contribution of grid human body classification satisfies the relationship: ; in, Indicates The contribution of each grid to human body classification. Indicates In the grid Line The pixel value of the column position. The greater the contribution, the greater the influence of the grid on human classification. The human classification contribution of each grid is calculated to obtain the human classification contribution sequence of the image. The tracking and positioning module 104 is used to calculate the distance between contribution sequences and adjust the camera angle based on the distance to achieve tracking and positioning of the target.

[0037] Among them, the distance between the human body classification contribution sequence of the previous and next frame images is calculated, and the distance between the human body classification contribution sequence of the previous and next frame images is compared with a preset threshold. If it is less than or equal to the preset threshold, a first signal is output, and if it is greater than the preset threshold, a second signal is output; in response to the first signal, the person in the image has not moved, and there is no need to adjust the camera angle; in response to the second signal, the person in the image has moved, and the camera angle needs to be adjusted.

[0038] Preferably, the threshold is set to S / 6, where S is the area size of the human body region.

[0039] If a person moves, the contribution of each grid in the image to the distance is used to determine which grids the human body moves in, and the angle of the camera is moved to move the center of the camera to the new human body center grid.

[0040] Specifically, if the human body classification contribution sequence distance between the previous and next frame images is greater than the set threshold, the human body in the image has moved. According to the contribution of each grid to the distance, the grids in which the human body area has moved are determined, and the grids with increased contribution to human body classification are extracted to obtain the direction of human body movement. The coordinates of the center point of these grids as a whole are calculated, and the camera angle is adjusted to the center point. After one adjustment is completed, the new grid division result of the image is determined based on the initial frame taken by the camera after the angle adjustment. When there is no human body area in the entire camera, that is, no human body area is detected within the range that can be shot at all angles of the camera, the angle of the camera is adjusted to the center position to prevent the camera from failing to detect the human body area after rotating to a certain angle and then shooting at extreme angles.

[0041] As can be seen from the above description of the modular design of the present invention, the system of the present invention can be flexibly arranged according to the application scenario or demand without being limited to the architecture shown in the accompanying drawings. Further, it should be understood that any module, unit, component, server, computer or device that performs the operation of the present invention example may include or otherwise access a computer-readable medium, such as a storage medium, a computer storage medium or a data storage device (removable) and / or non-removable) such as a disk, an optical disk or a tape. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data.

[0042] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0043] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A target tracking and positioning system based on video analysis, characterized in that: include: A data detection module is used to input the video data captured by the camera in different scenes into a pre-trained person recognition network model for detection to obtain a human body region feature image set; A feature generation module, used to calculate the portrait feature data of each image in the human body region feature image set, and divide the human body region feature map into grids based on the portrait feature data to obtain a human body grid feature map; A calculation module, used for calculating the contribution of each grid in the human body grid feature map to human body classification to construct a contribution sequence, wherein the human body grid feature map and the contribution sequence correspond one to one; The tracking and positioning module is used to calculate the distance between the contribution degree sequences of adjacent images in the human body area feature image set, and adjust the camera angle based on the distance to achieve tracking and positioning of the target.

2. The target tracking and positioning system based on video analysis according to claim 1, characterized in that: The portrait feature data includes the ratio of the area of ​​the human body circumscribed matrix to the entire image area, the ratio of the human body area in the external matrix, the average human body density in rows and the average human body density in columns, including: The ratio of the area of ​​the human body circumscribed rectangle to the entire image area satisfies the relationship: ； The proportion of the human body area in the external matrix satisfies the relationship: ； The average density of pedestrians satisfies the relationship: ； The average density of the human body in the column satisfies the relationship: ； in, It represents the ratio of the area of ​​the human body's circumscribed rectangle to the entire image area. represents the rows of the image, represents the columns of the image, Indicates the proportion of the human body area within the circumscribed rectangle. represents the average density of pedestrians, represents the average density of human body, Represents the area of ​​the human body in the circumscribed rectangle, represents the area of ​​the circumscribed rectangle, Indicates the width of the bounding rectangle. Indicates the height of the bounding rectangle.

3. The target tracking and positioning system based on video analysis according to claim 1, characterized in that: The human body region feature map is divided into grids including the number of row grids and the number of column grids, including: The number of row grids satisfies the relationship: ； The number of column grids satisfies the relationship: ； in, Represents the number of row grids of the image, Represents the number of column grids of the image, It represents the ratio of the area of ​​the human body's circumscribed rectangle to the entire image area. Indicates the proportion of the human body area within the circumscribed rectangle. represents the average density of pedestrians, Represents the average density of human body.

4. The target tracking and positioning system based on video analysis according to claim 1, characterized in that: The contribution of the grid human body classification satisfies the relationship: ； in, Indicates The contribution of each grid to human body classification. Indicates In the grid Line The pixel value at the column position.

5. The target tracking and positioning system based on video analysis according to claim 1, characterized in that: The adjusting the camera angle based on the distance comprises: Based on the contribution sequence, a distance between two adjacent contribution sequences is calculated, and the distance is compared with a preset threshold. If the distance is less than or equal to the preset threshold, a first signal is output; if the distance is greater than the preset threshold, a second signal is output; In response to the first signal, the person in the image does not move, and there is no need to adjust the camera angle; In response to the second signal, the person in the image moves and the camera angle needs to be adjusted.

6. The target tracking and positioning system based on video analysis according to claim 1, characterized in that: The method for adjusting the camera angle comprises: Based on the contribution changes of the grids at the same position in the adjacent frame images, the grids with changed character areas are extracted, and the grids with increased contribution are screened to obtain the moving direction of the character; The coordinates of the center point of the entire grid whose contribution becomes larger are calculated, and the camera is controlled to adjust to the center point.

Images