An intelligent monitoring method for indoor specific areas and a video monitoring terminal

Abstract

The present invention relates to the field of feature-based motion analysis technology, and specifically to a method for intelligently monitoring a specific indoor area and a video monitoring terminal. When the method is applied, possible indicators of furniture contours are obtained through the grayscale differences and gradient values ​​of pixels at the same position in grayscale images of different frame regions; initial correction coefficients are obtained based on the position distribution of pixels of each category in a preset local area of ​​pixels in each frame region grayscale image and possible indicators of furniture contours, as well as the similarity of grayscale differences with pixels at corresponding positions in adjacent frame images; and the initial correction coefficients are used to adjust possible indicators of furniture contours to obtain optimized weights; and each frame region grayscale image is monitored using an optical flow algorithm based on the optimized weights. The present invention obtains optimized weights based on the position characteristics and grayscale change characteristics of pixels in a local area of ​​furniture pixels that block moving children, thereby increasing target recognition accuracy and improving the accuracy of intelligent monitoring of specific indoor areas.

Description

Technical Field

[0001] The present invention relates to the technical field of feature-based motion analysis, and in particular to an intelligent monitoring method for a specific indoor area and a video monitoring terminal. Background Art

[0002] Parents can't always be with their children due to work, rest, or other reasons. Installing cameras in their homes allows them to monitor the safety of moving objects, such as children, the elderly, and pets, in real time. This helps them identify any unusual accidents involving moving objects and ensures that appropriate emergency measures can be taken promptly. Therefore, intelligent surveillance of specific indoor areas prone to dangerous accidents is crucial.

[0003] When monitoring specific indoor areas, optical flow is used to obtain the specific position of moving objects in video frames. The distance between the moving object's position and the danger zone is used to determine whether the moving object is in a dangerous state, thereby achieving surveillance of the specific area. Because indoor furniture such as tables, chairs, sofas, and doors are present, moving objects in specific areas may be obscured by the furniture. Therefore, the optical flow obtained by the optical flow method cannot accurately reflect the object's motion state. This can lead to target loss or misidentification of the moving object's position during the tracking process, reducing the accuracy of object recognition and, in turn, reducing the accuracy of intelligent monitoring of moving objects in specific indoor areas. Summary of the Invention

[0004] In order to solve the technical problem that the optical flow method has errors in identifying the position of moving objects blocked by furniture, resulting in inaccurate intelligent monitoring of specific indoor areas, the purpose of the present invention is to provide an intelligent monitoring method and video monitoring terminal for specific indoor areas. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a method for intelligently monitoring a specific indoor area, which is applied to a video surveillance terminal that communicates with a server in real time. The method includes:

[0006] Obtain continuous frame grayscale images of a specific indoor area;

[0007] Obtain possible furniture contour indicators for each pixel in each frame's regional grayscale image based on the grayscale difference and gradient value of the pixel at the same position in each frame's regional grayscale image and the grayscale images in the other frames;

[0008] categorize the pixels within a preset local area of ​​each pixel in each frame of the regional grayscale image; obtain an initial correction coefficient for each pixel in each frame of the regional grayscale image based on the position distribution of the pixels of each category within the preset local area of ​​each pixel in each frame of the regional grayscale image and the possible indicators of the furniture outline, as well as the grayscale value difference between the pixels of each category and the corresponding pixels in the adjacent frame of the regional grayscale image;

[0009] Based on the initial correction coefficient of the pixel points in the preset local area of ​​each pixel point in each frame of the regional grayscale image, the furniture outline possible index of each pixel point is adjusted to obtain the optimized weight of each pixel point in each frame of the regional grayscale image;

[0010] Based on the optimized weights, the optical flow algorithm is used to monitor the grayscale image of each frame region.

[0011] Furthermore, the method for obtaining possible furniture outline indicators for each pixel point in each frame of the regional grayscale image includes:

[0012] A frame of regional grayscale image is selected as the analysis image, and the remaining regional grayscale images except the analysis image in the regional grayscale image corresponding to the acquisition time within a preset time period of the acquisition time of the analysis image are used as the neighboring images of the analysis image; the monitoring background image of the corresponding regional grayscale image is constructed by the mean grayscale value of the pixel points at the same position in all neighboring images of each frame of regional grayscale image;

[0013] According to the grayscale value difference and gradient value of the pixel at the same position in each frame of the regional grayscale image and the monitoring background image, the possible indicator of the furniture outline of each pixel in each frame of the regional grayscale image is obtained.

[0014] Furthermore, the method of classifying pixel points within a preset local area of ​​each pixel point in each frame of the regional grayscale image includes:

[0015] The difference in grayscale values ​​of the pixels at the same position in each frame regional grayscale image and its adjacent frame regional grayscale image is recorded as the grayscale difference index of each pixel in each frame regional grayscale image; based on the grayscale difference index, the pixel points in the preset local area of ​​each pixel in each frame regional grayscale image are divided into categories.

[0016] Furthermore, the method of classifying pixel points within a preset local area of ​​each pixel point in each frame of regional grayscale image based on the grayscale difference index includes:

[0017] For each pixel point in each frame of the regional grayscale image, the pixel points in the preset local area of ​​each pixel point are clustered based on the grayscale difference index to obtain two clusters. All the pixels in each cluster are pixels of one category in the preset local area of ​​each pixel point.

[0018] Furthermore, the method for obtaining the initial correction coefficient of each pixel in each frame of the regional grayscale image includes:

[0019] The pixel points on the boundary line of the two categories of pixel points in the preset local area of ​​each pixel point in each frame of the regional grayscale image are recorded as the dividing points in the preset local area; based on the possible furniture contour index of the dividing point in the preset local area of ​​each pixel point in the regional grayscale image, and the gradient value of the pixel point at the corresponding position of the dividing point in the monitoring background image, the real furniture contour degree of each pixel point in each frame of the regional grayscale image is obtained;

[0020] Obtaining a similarity index of a corresponding category based on the discrete degree of the grayscale difference index of each category of pixels within a preset local area of ​​each pixel in each frame of the regional grayscale image;

[0021] The initial correction coefficient of each pixel point in each frame of the regional grayscale image is obtained by combining the real furniture contour of each pixel point in each frame, the total number of connected domains formed by pixel points of all categories in the preset local area of ​​the pixel point, and the similarity index.

[0022] Furthermore, the method for obtaining the optimized weight of each pixel in each frame of the regional grayscale image includes:

[0023] Selecting a pixel point from each frame of the regional grayscale image as the target pixel point, and selecting a furniture category from all categories within a preset local area of ​​the target pixel point based on the grayscale difference index; determining whether the target pixel point belongs to the pixel point of the furniture category; if so, calculating the average of the initial correction coefficients of all the dividing points within the preset local area of ​​the target pixel point, and summing the average and the initial correction coefficient of the target pixel point as the final correction coefficient of the target pixel point; if not, using the initial correction coefficient of the target pixel point as the final correction coefficient;

[0024] The final correction coefficient of the target pixel point is used to adjust the possible index of the furniture outline to obtain the optimized weight of the target pixel point.

[0025] Furthermore, the method for monitoring each frame of regional grayscale image using an optical flow algorithm based on the optimized weights includes:

[0026] The optimized weight of the pixel points in each frame of the regional grayscale image is used as the weight of each pixel point in the least square method of the optical flow algorithm; and the improved optical flow algorithm is used to monitor each frame of the regional grayscale image.

[0027] Furthermore, the optical flow algorithm is an LK optical flow algorithm.

[0028] Furthermore, an implementation method of selecting a furniture category from all categories within a preset local area of ​​a target pixel based on the grayscale difference index includes:

[0029] For each category in the preset local area of ​​the target pixel point, the mean value of the grayscale difference index of the pixel points of each category is recorded as the judgment index of the corresponding category, and the category corresponding to the smallest judgment index is selected as the furniture category of the target pixel point.

[0030] In the second aspect, another embodiment of the present invention also provides a video surveillance terminal, including a processor and a memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call the program instructions in the memory to execute the above-described method.

[0031] The present invention has the following beneficial effects:

[0032] In an embodiment of the present invention, the grayscale difference and gradient value of pixels at the same position in each frame of the regional grayscale image and the remaining regional grayscale images are used to analyze the furniture outline features in the image. In the local area of ​​the pixel at the furniture outline position, some pixels have obvious grayscale changes in adjacent frame images, while others have insignificant grayscale changes. Therefore, it is necessary to classify the pixels in the preset local area of ​​the pixel. The pixels in each category represent the same object, namely, furniture or children. The pixels in the same category are distributed at a distance and have similar grayscale changes. The above factors are combined to obtain an initial correction coefficient for the pixel. The initial correction coefficient reflects the possibility that the pixel appears to be a furniture outline pixel that obscures the child. Since the furniture itself can also obstruct the movement of the child, the initial correction coefficient of the pixels in the preset local area of ​​each pixel reflects the possibility that the local position of the pixel is a furniture outline. The initial correction coefficient is used to adjust the furniture outline possibility index of the pixel to obtain an optimized weight and analyze the possibility that the furniture outline obscures the child. The optical flow algorithm is optimized based on the optimized weight to improve the accuracy of the optical flow algorithm in tracking moving objects in the regional grayscale image, reduce the probability of misidentification or target loss, and improve the accuracy of intelligent monitoring of moving objects in specific indoor areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 A flowchart of a method for intelligently monitoring a specific indoor area according to an embodiment of the present invention;

[0035] Figure 2 A flowchart of a method for obtaining initial correction coefficients for pixels in a regional grayscale image of each frame provided by one embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of the hardware structure of a video surveillance terminal provided by one embodiment of the present invention;

[0037] Figure 4 This is a structural diagram of an indoor specific area intelligent monitoring system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0038] To further illustrate the technical means and effectiveness of the present invention in achieving its intended objectives, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of an intelligent indoor area monitoring method and video surveillance terminal proposed by the present invention. In the following description, references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0039] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0040] The present invention is directed to a specific scenario in which, when a camera is used to monitor children indoors, furniture blocks part of the child's area, causing the camera to lose its tracking target, i.e., the moving child.

[0041] The following describes in detail a specific solution of an indoor specific area intelligent monitoring method and a video monitoring terminal provided by the present invention with reference to the accompanying drawings.

[0042] Example 1:

[0043] Figure 1The present invention is a flowchart of a method for intelligently monitoring a specific indoor area. The method can be applied to a video surveillance terminal that communicates with a server in real time. The method includes:

[0044] Step S1: Acquire continuous frame grayscale images of a specific indoor area.

[0045] Specifically, this solution uses children in motion as an example for subsequent analysis. A high-definition camera is installed in a specific area indoors. It captures real-time images of children's movements within this area, generating continuous frame images of the region. Each frame of the region image is an RGB image. Because the time interval between adjacent frames is small, the illumination in the region images of adjacent frames can be assumed to be the same, meeting the constant illumination assumption. Therefore, an optical flow algorithm can be used to analyze the movement of children obscured by furniture. To speed up the algorithm's runtime and provide real-time information, the region images are grayscaled to obtain grayscale images, which are then used to analyze the child's movement.

[0046] It should be noted that in the embodiments of the present invention, a weighted average grayscale algorithm is used for grayscale processing. The weighted average grayscale algorithm is a well-known technical means for those skilled in the art. The acquisition time of the regional grayscale image is the moment when the camera captures the corresponding regional image of the regional grayscale image. The acquisition time interval between two adjacent frames of regional grayscale images is 0.1 seconds, which can be set by the implementer according to specific circumstances. Other image acquisition devices and image preprocessing algorithms may also be used in other embodiments of the present invention. Both image acquisition and image preprocessing algorithms are well-known technical means for those skilled in the art and are not limited here.

[0047] Step S2: Obtain possible furniture contour indicators for each pixel in each frame of regional grayscale image based on the grayscale difference and gradient value of the pixel at the same position in each frame of regional grayscale image and the remaining frame of regional grayscale images.

[0048] Assuming constant illumination between adjacent frames, optical flow algorithms can be widely used to detect and track children in surveillance videos based on the consistent movement of pixels within a window. When a child is obscured by furniture, some of the child's pixels are directly covered by the furniture pixels. This causes the motion direction of the pixels in the obscured and unobstructed parts of the child to be inconsistent, resulting in large errors in the optical flow direction calculated using the least squares method. Therefore, it is necessary to adjust the least squares weights used in the optical flow algorithm based on whether the child's pixels are obscured by furniture. By weighting the pixels, the influence of furniture outlines in the image is reduced, thereby improving the accuracy of the optical flow analysis.

[0049] In this solution, objects that may obstruct children, such as tables, chairs, sofas, doors, beds, and walls, are collectively referred to as furniture. Due to the difference in the front-to-back position of the furniture and the child in space, the furniture in the regional grayscale image obscures the child, causing the boundary of the obscured portion of the child to be represented as the furniture outline. Furthermore, the significant color difference between the furniture and the child results in significant gradient variations in the pixels at the boundary of the portion of the child obscured by the furniture. Therefore, the gradient features of the pixels at the same position in the regional grayscale image and the background can be used to determine the likelihood that the pixel represents a furniture outline. Pixels with larger gradient values ​​are more likely to represent furniture outlines that obscure the child. To minimize the impact of these pixels on the optical flow algorithm, they should be given a smaller weight.

[0050] Because the boundary between the child and the background also exhibits a large gradient, the aforementioned method also assigns a smaller weight to pixels representing the child's outline, causing the optical flow algorithm to ignore changes in the child and reducing child tracking accuracy. While the furniture is fixed during monitoring, the child's position is not. The grayscale difference and gradient value between each frame's regional grayscale image and the pixel at the same position in the background reflect the likelihood that the pixel is located at the furniture's outline, thereby obtaining a possible indicator of the furniture's outline. The overall grayscale level of pixels at the same position in multiple frames of the regional grayscale image represents the background, minimizing interference from the child's movement.

[0051] Step S210: Select any frame of regional grayscale image as the analysis image, and use the remaining regional grayscale images except the analysis image in the regional grayscale image corresponding to the acquisition time within the preset time period of the acquisition time of the analysis image as the neighboring images of the analysis image; construct the monitoring background image of the corresponding regional grayscale image by the mean grayscale value of the pixel points at the same position in all neighboring images of each frame of the regional grayscale image.

[0052] It should be noted that since this solution uses a camera to monitor children's activities indoors in real time, the analysis image in this embodiment is acquired at the end of its preset time period. The duration of the preset time period is set to 1 minute as an empirical value and can be customized by the implementer based on specific circumstances. The neighboring images of the analysis image are acquired in each frame of the regional grayscale image using the same acquisition method.

[0053] If a pixel in a certain frame area grayscale image represents a moving child pixel, and the child moves in continuous frame images, then the pixel at the corresponding position of this pixel in the remaining frame area grayscale images may represent a background pixel. This scheme compares each frame area grayscale image with the background and analyzes the possibility that the pixel in the image is located at the outline of the furniture. Therefore, it is necessary to obtain a monitoring background image.

[0054] In another embodiment of the present invention, at least two frames of regional grayscale images obtained in a specific indoor area without children can be selected as initial background images, and the monitoring background image can be constructed by averaging the grayscale values ​​of pixels at the same position in all initial background images. It should be noted that in this embodiment, the monitoring background image is the same for each frame of the regional grayscale image.

[0055] It should be noted that in this scheme, the pixel points in any two frames of regional grayscale images correspond one to one, that is, the pixel coordinates of the corresponding pixel points in the same position of any two frames of regional grayscale images in their respective regional grayscale images are the same; and the pixel points in each frame of regional grayscale image and its monitored background image also correspond one to one.

[0056] Step S220: Obtain possible furniture contour indicators for each pixel in each frame of the regional grayscale image based on the grayscale value difference and gradient value of the pixel at the same position in each frame of the regional grayscale image and the monitoring background image.

[0057] Because the furniture is fixed in position during surveillance, while the children's positions are not, the grayscale values ​​of the pixels at the furniture's outline in each frame's regional grayscale image and the background image remain essentially constant, with large gradient values. However, the grayscale values ​​of the pixels at the moving children differ due to their movement, and the gradient values ​​in the background image are smaller. Therefore, the grayscale value difference between the pixels at the same location in each frame's regional grayscale image and the background image is negatively correlated with the furniture's outline potential indicator, while the gradient value is positively correlated with the furniture's outline potential indicator.

[0058] Preferably, in an embodiment of the present invention, the method for obtaining possible indicators of furniture contours is: negatively correlated mapping the absolute value of the difference between the grayscale values ​​of each frame of regional grayscale image and the pixel points at the same position in its monitored background image to obtain the grayscale index of each pixel point in each frame of regional grayscale image; recording the sum of the gradient values ​​of each frame of regional grayscale image and the pixel points at the same position in its monitored background image as the gradient index of each pixel point in each frame of regional grayscale image; and taking the product of the grayscale index and the gradient index as the possible indicator of furniture contours for each pixel point in each frame of regional grayscale image.

[0059] It should be noted that in this embodiment of the present invention, the exp(-) function is used to perform a negative correlation mapping between the absolute difference between the grayscale values ​​of the pixels at the same location in each frame of the regional grayscale image and its monitored background image. In other embodiments, the negative correlation mapping can be achieved by taking the inverse, which is not limited or elaborated here. Here, exp is an exponential function with the natural constant e as its base.

[0060] Step S3: Divide the pixel points in the preset local area of ​​each pixel point in each frame regional grayscale image into categories; obtain the initial correction coefficient of each pixel point in each frame regional grayscale image based on the position distribution of the pixel points of each category in the preset local area of ​​each pixel point in each frame regional grayscale image and the possible indicators of the furniture outline, as well as the similarity between the pixel points of each category and the grayscale value difference of the pixel points at the corresponding position in the adjacent frame regional grayscale image of each frame regional grayscale image.

[0061] If the grayscale of the local area at the same position in the grayscale image of the adjacent frame shows a significant change, then the local area represents the area of ​​the moving child. Although the grayscale of the furniture pixels that appear to be occluding the child shows a significant change in the grayscale in the adjacent frame images, because only part of the child is occluded by the furniture, not all pixels in the local area will show a grayscale change. Some pixels have grayscale values ​​similar to those of the corresponding pixels in the previous frame image. As a result, the pixels in the local area of ​​the furniture pixels that appear to be occluding the child can be divided into two categories: those with significant grayscale changes and those with insignificant grayscale changes. Therefore, the pixels in the preset local area of ​​each pixel in each frame of the regional grayscale image are divided into categories.

[0062] The method for categorizing pixels within a preset local area includes: recording the grayscale value difference between each frame's regional grayscale image and the pixel at the same location in its adjacent frame's regional grayscale image as a grayscale difference index for each pixel in each frame's regional grayscale image; and categorizing the pixels within the preset local area for each pixel in each frame's regional grayscale image based on the grayscale difference index. It should be noted that the adjacent frame's regional grayscale image of each frame's regional grayscale image is the previous frame's regional grayscale image; the size of the preset local area is 7×7, with the pixel located at the center of the preset local area, which can be customized by the implementer based on specific circumstances; the preset local area can also be replaced with a preset local window.

[0063] Preferably, in an embodiment of the present invention, for each pixel in each frame of the regional grayscale image, the pixels within the preset local region of each pixel are clustered based on the grayscale difference index to obtain two clusters, and all pixels within each cluster belong to one category of pixels within the preset local region of each pixel. It should be noted that in an embodiment of the present invention, the K-means clustering algorithm is selected for clustering, where K = 2, i.e., there are two categories of pixels within the preset local region of each pixel, one category of furniture pixels and the other category of moving children pixels.

[0064] In another embodiment of the present invention, the mean of the grayscale difference index of each pixel within a preset local area is recorded as the benchmark segmentation value of each pixel; pixels within the preset local area where the grayscale difference index is less than the benchmark segmentation value are classified as pixels of the first category, and pixels where the grayscale difference index is greater than or equal to the benchmark segmentation value are classified as pixels of the second category. The mean, median, and mode can all reflect the central tendency of a set of data, and the mean can also be replaced by the median or mode.

[0065] Not all furniture blocks children, so the furniture outline indicators need to be corrected to improve the accuracy of child tracking. Therefore, the furniture outline indicators are corrected by using the grayscale changes of the corresponding pixels in the adjacent frame images within the preset local area of ​​pixels.

[0066] The pixels in the local area of ​​the furniture pixels that appear to be occluding children can be divided into two categories: those with obvious grayscale changes and those with inconspicuous grayscale changes. Since the pixels in each category represent the same object, the position distribution of the pixels in the same category is concentrated. The closer the grayscale changes of the pixels in the same category in adjacent frame images are, the easier it is for the pixels in the preset local area of ​​the pixels to be divided into two categories, and the greater the possibility that the pixels appear to be furniture pixels that appear to be occluding children. However, some of the pixels that appear to be children may also have similarities before and after movement, such as black hair, skin color, and other factors interfering, resulting in the pixels in the preset local area being clearly divided into two categories. Therefore, the boundary pixels that appear as category boundaries in the two categories of pixels should appear as furniture contour pixels. The initial correction coefficient of the possible indicators of the furniture contour is obtained by combining the above factors.

[0067] See also Figure 2 , which shows a flowchart of the steps of a method for obtaining initial correction coefficients of pixels in a regional grayscale image of each frame provided by an embodiment of the present invention, the method comprising:

[0068] Step S310: The pixel points on the boundary line of the two categories of pixel points in the preset local area of ​​each pixel point in each frame of the regional grayscale image are recorded as the dividing points in the preset local area; based on the possible indicators of the furniture contour of the dividing points in the preset local area of ​​each pixel point in each frame of the regional grayscale image, and the gradient value of the pixel points at the corresponding position of the dividing points in the monitoring background image, the real furniture contour degree of each pixel point in each frame of the regional grayscale image is obtained.

[0069] The greater the furniture outline likelihood index of a pixel point on the boundary line between two categories of pixels within the preset local region of a pixel point, i.e., the dividing point, and the greater the gradient value of the pixel point at the corresponding location of the dividing point in the monitored background image, the greater the likelihood that the boundary line within the preset local region of the pixel point is a furniture outline, and the greater the likelihood that the pixel point is located at a furniture outline location. Therefore, the furniture outline likelihood index of each dividing point and the gradient value of the pixel point at the corresponding location of the dividing point in the monitored background image are positively correlated with the true furniture outline degree.

[0070] Preferably, in an embodiment of the present invention, the method for obtaining the real furniture contour includes: selecting any pixel point from each frame of the regional grayscale image as the analysis pixel point, selecting any category from the preset local area of ​​the analysis pixel point as the analysis category, and taking another category as the target category; for the pixel point of the analysis category, if there is a pixel point of the target category within the preset window of each pixel point, then the pixel point is used as the demarcation point within the preset local area of ​​the analysis pixel point; the product of the possible furniture contour index of each demarcation point within the preset local area of ​​each pixel point in each frame of the regional grayscale image and the gradient value of the pixel point at the corresponding position of the demarcation point in the monitoring background image is used as the initial furniture contour of the corresponding demarcation point; the average of the initial furniture contours of all demarcation points within the preset local area of ​​each pixel point in each frame of the regional grayscale image is used as the real furniture contour of the corresponding pixel point.

[0071] It should be noted that the size of the preset pixel window is smaller than the size of the preset local area. The empirical value of the preset pixel window size is 3×3, and the pixel is located at the center of the preset window. Implementers can set this size based on specific circumstances. The method for obtaining the demarcation point within the preset local area for each pixel in each frame of the regional grayscale image is the same as the method for obtaining the demarcation point within the preset local area of ​​the analysis pixel.

[0072] Step S320: obtaining a similarity index of a corresponding category based on the discrete degree of the grayscale difference index of each category of pixels within a preset local area of ​​each pixel in each frame of the regional grayscale image.

[0073] If the grayscale difference index of each category of pixels in the preset local area of ​​the pixel point is more similar, it means that it is easier to classify the pixels in the preset local area based on the motion situation, and the division effect of the two categories in the preset local window is better, the two categories are more likely to represent furniture pixels and children's pixels, and the pixel point is more likely to be located at the outline of furniture that blocks the child.

[0074] Preferably, in an embodiment of the present invention, the method for obtaining the similarity index is: performing a negative correlation mapping on the variance of the grayscale difference index of all pixels of each category within a preset local area of ​​each pixel in each frame of the regional grayscale image to obtain the similarity index of the corresponding category. It should be noted that this embodiment measures the degree of similarity by variance. The smaller the variance, the more concentrated the grayscale difference index of the pixels of each category and the greater the degree of similarity. The negative correlation mapping is performed using the exp(-) function, and the negative correlation mapping can also be achieved by taking the inverse, which is not limited or elaborated here.

[0075] The standard deviation and the range both reflect the degree of dispersion of the data. In another embodiment of the present invention, the variance can be replaced by the standard deviation or the range.

[0076] Step S330: Combine the actual furniture contour of each pixel point in each frame of the regional grayscale image, and the total number and similarity index of connected domains formed by pixels of all categories in the preset local area of ​​the pixel point to obtain the initial correction coefficient of each pixel point in each frame of the regional grayscale image.

[0077] The distribution of two types of pixels in the preset local area of ​​furniture pixels that appear to be obstructing children is concentrated, that is, pixels that appear to show obvious grayscale changes when children are moving can form a connected area, and pixels that appear to show non-obvious grayscale changes when furniture is also formed. Therefore, the closer the total number of connected domains formed by all categories of pixels in the preset local area of ​​the pixel is to the constant 2, the greater the possibility that the pixel is located at the outline of furniture that obstructs children. The greater the true furniture outline of the pixel and the greater the similarity index of each category in its preset local area, the greater the possibility that the pixel is located at the pixel of the furniture outline that obstructs children. Therefore, both the true furniture outline and the similarity index are positively correlated with the initial correction coefficient, and the difference between the total number of connected domains formed by all categories of pixels in the preset local area of ​​the pixel and the constant 2 is positively correlated with the initial correction coefficient.

[0078] Preferably, in an embodiment of the present invention, the method for obtaining the initial correction coefficient is: for each pixel point in each frame of the regional grayscale image, the cumulative sum of the similarity indices of all categories in the preset local area of ​​each pixel point is used as the comprehensive similarity value of the corresponding pixel point; the pixels of each category in the preset local area of ​​each pixel point respectively constitute a connected domain, and the total number of connected domains in the preset local area is counted; the absolute value of the difference between the total number of connected domains in the preset local area of ​​each pixel point and a preset positive number is calculated and negatively correlated with each other to perform mapping, which is used as the connected domain index of the corresponding pixel point; the product of the comprehensive similarity value of each pixel point in each frame of the regional grayscale image, the connected domain index and the real furniture contour is used as the initial correction coefficient of the corresponding pixel point.

[0079] It should be noted that, since the pixels in the preset local area of ​​the furniture outline pixels that block the children can be clearly divided into two categories, the preset positive number in the embodiment of the present invention takes the empirical value 2, and the implementer can set it according to the specific situation; by taking the inverse negative correlation mapping of the absolute value of the difference between the total number of connected domains in the preset local area of ​​each pixel point and the preset positive number, no limitation or elaboration is made here.

[0080] Step S4: Based on the initial correction coefficient of the pixel points in the preset local area of ​​each pixel point in each frame of the regional grayscale image, the furniture contour possible index of each pixel point is adjusted to obtain the optimized weight of each pixel point in each frame of the regional grayscale image.

[0081] Furniture outlines aren't the only thing that can obstruct children's movement; the furniture itself can also obstruct them. Since furniture itself lacks gradients and grayscale variations, it plays a significant role in the optical flow algorithm, necessitating a correction to the initial correction coefficients. The initial correction coefficients for each pixel within a preset local area reflect the likelihood that the pixel's local position is a furniture outline. This coefficient is then used to adjust the pixel's furniture outline likelihood indicator to obtain an optimized weight.

[0082] Step S410: Select any pixel point from each frame of the regional grayscale image as the target pixel point, and select the furniture category from all categories in the preset local area of ​​the target pixel point based on the grayscale difference index; determine whether the target pixel point belongs to the furniture category pixel point, if so, calculate the mean of the initial correction coefficients of all dividing points in the preset local area of ​​the target pixel point, and use the sum of the mean and the initial correction coefficient of the target pixel point as the final correction coefficient of the target pixel point; if not, use the initial correction coefficient of the target pixel point as the final correction coefficient.

[0083] Since the position of furniture is fixed during monitoring, while the position of children is not, and the position of moving children changes in adjacent frame images, the local area with obvious grayscale changes at the same position in the grayscale image of the adjacent frame area represents the child, while the local area without grayscale changes represents the furniture. Therefore, the mean of the grayscale difference index of all pixels of each category in the preset local area of ​​the target pixel is calculated, and the category corresponding to the smallest mean is taken as the furniture category. If a pixel belongs to the furniture category in its preset local area, it means that the pixel is a pixel that blocks the moving child, and the optical flow obtained by the optical flow algorithm cannot accurately reflect the movement state of the child. The initial correction coefficient of such pixel needs to be adjusted; if the pixel belongs to the child category, the furniture does not cause an error in the optical flow calculation of the pixel, and the initial correction coefficient of such pixel is directly used as the final correction coefficient.

[0084] It should be noted that if a pixel point belongs to a pixel point of the furniture category within its preset local area, the larger the initial correction coefficient of the demarcation point within the preset local area of ​​the pixel point, the greater the possibility that the demarcation point is located at the outline of furniture that blocks the child, and the greater the degree to which the pixel point needs to be adjusted due to furniture occlusion. Therefore, this scheme uses the initial correction coefficient of the demarcation point within the preset local area of ​​the target pixel point to adjust the initial correction coefficient of the target pixel point; specifically, the initial correction coefficient of the target pixel point is adjusted by adding the initial correction coefficient of all demarcation points within the preset local area of ​​the target pixel point to the average of the initial correction coefficients of all demarcation points within the preset local area of ​​the target pixel point.

[0085] Step S420: Using the final correction coefficient of the target pixel point, the furniture outline possible index is adjusted to obtain the optimized weight of the target pixel point.

[0086] If the final correction coefficient of the pixel point and the furniture contour possible index are both larger, it means that the pixel point represents the furniture contour pixel point and is more likely to occlude the moving child. The greater the possibility of the pixel point representing the furniture contour pixel occluding the child, the greater the error of the optical flow obtained by the optical flow algorithm in reflecting the child's movement state. Therefore, when analyzing the optical flow algorithm, the weight of the child's occluded pixel point should be smaller.

[0087] Preferably, in an embodiment of the present invention, the method for obtaining the optimization weight is: negatively correlating and normalizing the final correction coefficient of the target pixel point with the product of the possible index of the furniture outline to obtain the optimization weight of the target pixel point.

[0088] It should be noted that in the embodiment of the present invention, negative correlation and normalization can be performed using the exp(1-x) function. Alternatively, the product of the final correction coefficient and the possible furniture outline index can be normalized, and the difference between the normalized result and a constant can be subtracted to use as the optimization weight. Other basic mathematical operations can also be used, which are not limited or elaborated here. Wherein, x is an independent variable, which can be the product of the final correction coefficient and the possible furniture outline index.

[0089] In the embodiment of the present invention, the Norm function is used to normalize the product of the final correction coefficient and the possible index of the furniture contour. In the embodiment of the present invention, other normalization methods can also be selected, such as function transformation, maximum and minimum normalization, etc., which are not limited here.

[0090] Step S5: Monitor the regional grayscale image of each frame using the optical flow algorithm based on the optimized weights.

[0091] In the optical flow algorithm, the least squares method is used to minimize the difference between pixels in a window around a pixel to find the best motion estimate. In the process of using the optical flow algorithm to obtain the optical flow vector of a feature point, the ordinary least squares solution treats all pixels in the window of the feature point equally, that is, assigns the same weight to all pixels in the window. In order to improve the overall motion estimation accuracy and reliability, this solution should assign higher weights to the child pixels that are not blocked by furniture in the calculation, and assign lower weights to the furniture pixels that block the moving child. The formula of the optical flow algorithm is modified in a weighted manner using the optimized weights. The optimized weights of the pixels in each frame of the regional grayscale image are used as the weights of each pixel in the least squares method of the optical flow algorithm, so that the weighted error is minimized to obtain the accurate motion direction. The improved optical flow algorithm is used to monitor the regional grayscale image of each frame, and the same method is used to achieve target tracking of the child.

[0092] It should be noted that in the embodiment of the present invention, the LK (Lucas-Kanade) optical flow algorithm is selected to analyze the motion of objects in each frame of the regional grayscale image. In other embodiments of the present invention, other optical flow algorithms, such as the KLT (Kanade-Lucas-Tomasi) optical flow algorithm, may also be selected. The LK optical flow algorithm is a formula technology known to those skilled in the art and will not be described in detail here.

[0093] The optical flow method is used to track the moving children in the grayscale images of continuous frames, and the child position area in each frame of the grayscale image is obtained. Since the dangerous area and the camera position are fixed, the dangerous area can be directly located at the same position in each frame of the grayscale image, and the pixel points can be directly pre-selected as the dangerous position representation points. The Euclidean distance between the center of mass of the child position area in each grayscale video frame and the representative point of each dangerous position is calculated, and recorded as the judgment distance of each dangerous position representative point; if the judgment distance is less than the warning distance, a reminder is issued through a mobile phone text message, APP, or abnormal ringtone, thereby realizing intelligent monitoring of children in specific indoor areas. It should be noted that the dangerous areas are the balcony, kitchen and other areas.

[0094] So far, the present invention is completed.

[0095] In summary, in an embodiment of the present invention, possible indicators of furniture contours are obtained through the grayscale differences and gradient values ​​of pixels at the same position in grayscale images of different frame regions; an initial correction coefficient is obtained based on the position distribution of pixels of each category in a preset local area of ​​pixels in each frame region grayscale image and the possible indicators of furniture contours, as well as the degree of similarity with the grayscale differences of pixels at corresponding positions in grayscale images of adjacent frame regions; and the possible indicators of furniture contours are adjusted using the initial correction coefficient to obtain optimized weights; and the optical flow algorithm is used to monitor the grayscale images of each frame region based on the optimized weights. The present invention obtains optimized weights based on the position characteristics and grayscale change characteristics of pixels in a preset local area of ​​furniture pixels that occlude moving children, thereby improving the accuracy of intelligent monitoring of specific indoor areas.

[0096] Example 2:

[0097] Figure 3 FIG. 1 is a schematic diagram of the hardware structure of a video surveillance terminal provided by an embodiment of the present invention. Figure 3 As shown, the video surveillance terminal includes: a processor 601 and a memory 602 and a bus 603 connected to the processor 601; wherein the processor 601 and the memory 602 communicate with each other through the bus 603; the processor 601 is used to call the program instructions in the memory 602 to execute the above method.

[0098] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to execute an intelligent monitoring method for a specific indoor area provided by an embodiment of the present application.

[0099] In this embodiment, the device can be divided into functional modules based on the above-described method examples. For example, each functional module can be mapped to a specific functional module, or two or more functions can be integrated into a single processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and represents only a logical functional division. In actual implementation, other division methods may be used.

[0100] It should be understood that the device provided in this embodiment is used to execute the above-mentioned method for intelligently monitoring a specific indoor area, and thus can achieve the same effect as the above-mentioned implementation method.

[0101] When an integrated unit is used, the device may include a processing module and a storage module. When the device is applied to a video surveillance terminal, the processing module may be used to control and manage the actions of the video surveillance terminal. The storage module may be used to support the video surveillance terminal in executing mutual program codes, etc.

[0102] The processing module may be a processor or controller that implements or executes various exemplary logic blocks, modules, and circuits disclosed herein. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor (DSP) and a microprocessor, and the storage module may be a memory.

[0103] Example 3:

[0104] This invention proposes an indoor specific area intelligent monitoring system, please refer to Figure 4 , which shows a structural diagram of an indoor specific area intelligent monitoring system provided by one embodiment of the present invention, the system includes:

[0105] The data acquisition module 710 is used to obtain continuous frame grayscale images of a specific indoor area.

[0106] The furniture outline analysis module 720 is used to obtain possible furniture outline indicators for each pixel in each frame of regional grayscale image based on the grayscale difference and gradient value of the pixel at the same position in each frame of regional grayscale image and the other frames of regional grayscale image.

[0107] The correction analysis module 730 is used to classify the pixel points in the preset local area of ​​each pixel point in each frame of the regional grayscale image; based on the position distribution of the pixel points of each category in the preset local area of ​​each pixel point in each frame of the regional grayscale image and the possible indicators of the furniture outline, as well as the similarity between the pixel points of each category and the grayscale value difference of the pixel points at the corresponding position in the adjacent frame regional grayscale images of each frame of the regional grayscale image, the initial correction coefficient of each pixel point in each frame of the regional grayscale image is obtained.

[0108] The priority weight analysis module 740 is used to adjust the furniture contour possible index of each pixel point based on the initial correction coefficient of the pixel point in the preset local area of ​​each pixel point in each frame of the regional grayscale image, and obtain the optimized weight of each pixel point in each frame of the regional grayscale image.

[0109] The monitoring optimization module 750 is used to monitor each frame of the regional grayscale image using an optical flow algorithm based on the optimization weight.

[0110] It should be noted that the devices provided in the above embodiments are merely illustrative of the division of the functional modules described above. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to perform all or part of the functions described above. Furthermore, the intelligent monitoring system for a specific indoor area provided in the above embodiments and the intelligent monitoring method for a specific indoor area provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiments and will not be repeated here.

[0111] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0112] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An intelligent monitoring method for a specific indoor area, characterized in that: A video surveillance terminal used for real-time communication with a server, the method comprising: Obtaining continuous frame area grayscale images of a specific indoor area; According to the grayscale difference and gradient value of the pixel points at the same position in each frame of the regional grayscale image and the other frame of the regional grayscale image, the possible index of the furniture outline of each pixel point in each frame of the regional grayscale image is obtained; Divide the pixel points in the preset local area of ​​each pixel point in each frame of regional grayscale image into categories; obtain the initial correction coefficient of each pixel point in each frame of regional grayscale image according to the position distribution of the pixel points of each category in the preset local area of ​​each pixel point in each frame of regional grayscale image and the possible indicators of the furniture contour, and the similarity between the pixel points of each category and the grayscale value difference of the pixel points at the corresponding position in the adjacent frame of regional grayscale image of each frame of regional grayscale image; Based on the initial correction coefficient of the pixel point in the preset local area of ​​each pixel point in each frame of the regional grayscale image, the possible index of the furniture outline of each pixel point is adjusted to obtain the optimization weight of each pixel point in each frame of the regional grayscale image; Based on the optimization weight, the regional grayscale image of each frame is monitored using an optical flow algorithm; The method for obtaining possible indicators of furniture contours for each pixel point in each frame of regional grayscale image includes: A frame of regional grayscale image is selected as the analysis image, and the remaining regional grayscale images except the analysis image in the regional grayscale image corresponding to the acquisition time within the preset time period of the acquisition time of the analysis image are used as the neighboring images of the analysis image; the monitoring background image of the corresponding regional grayscale image is constructed by the grayscale value average of the pixel points at the same position in all neighboring images of each frame of the regional grayscale image; Obtain possible furniture contour indicators for each pixel in each frame of the regional grayscale image according to the grayscale value difference and gradient value of the pixel at the same position in each frame of the regional grayscale image and the monitoring background image; The method for classifying pixel points in a preset local area of ​​each pixel point in each frame of regional grayscale image comprises: The difference in grayscale values ​​of pixels at the same position in each frame of the regional grayscale image and its adjacent frame of the regional grayscale image is recorded as a grayscale difference index of each pixel in each frame of the regional grayscale image; and the pixel points in a preset local area of ​​each pixel in each frame of the regional grayscale image are divided into categories based on the grayscale difference index; The method for obtaining the initial correction coefficient of each pixel in each frame of the regional grayscale image includes: The pixel points on the boundary line of the two categories of pixel points in the preset local area of ​​each pixel point in each frame of the regional grayscale image are recorded as the demarcation points in the preset local area; according to the possible indicators of the furniture contour of the demarcation points in the preset local area of ​​each pixel point in each frame of the regional grayscale image, and the gradient value of the pixel point at the corresponding position of the demarcation point in the monitoring background image, the real furniture contour degree of each pixel point in each frame of the regional grayscale image is obtained; Based on the discrete degree of the grayscale difference index of each category of pixel points in a preset local area of ​​each pixel point in each frame of the regional grayscale image, a similarity index of the corresponding category is obtained; The initial correction coefficient of each pixel in each frame of the regional grayscale image is obtained by combining the real furniture contour of each pixel in each frame of the regional grayscale image, the total number of connected domains formed by pixels of all categories in the preset local area of ​​the pixel and the similarity index.

2. The method for intelligently monitoring a specific indoor area according to claim 1, characterized in that: The method for classifying pixel points in a preset local area of ​​each pixel point in each frame of regional grayscale image based on the grayscale difference index comprises: For each pixel in each frame of the regional grayscale image, the pixels in the preset local area of ​​each pixel are clustered based on the grayscale difference index to obtain two clusters, and all the pixels in each cluster are pixels of one category in the preset local area of ​​each pixel.

3. The method for intelligently monitoring a specific indoor area according to claim 1, characterized in that: The method for obtaining the optimization weight of each pixel in each frame of the regional grayscale image includes: Select a pixel point from each frame of the regional grayscale image as the target pixel point, and select the furniture category from all categories in the preset local area of ​​the target pixel point based on the grayscale difference index; determine whether the target pixel point belongs to the pixel point of the furniture category, if so, calculate the mean of the initial correction coefficients of all the dividing points in the preset local area of ​​the target pixel point, and use the sum of the mean and the initial correction coefficient of the target pixel point as the final correction coefficient of the target pixel point; if not, use the initial correction coefficient of the target pixel point as the final correction coefficient; The final correction coefficient of the target pixel is used to adjust the possible index of the furniture outline to obtain the optimization weight of the target pixel.

4. The method for intelligently monitoring a specific indoor area according to claim 1, characterized in that: The method for monitoring each frame of regional grayscale image by using an optical flow algorithm based on the optimization weights includes: The optimized weight of the pixel points in each frame of the regional grayscale image is used as the weight of each pixel point in the least square method of the optical flow algorithm; and the improved optical flow algorithm is used to monitor each frame of the regional grayscale image.

5. The method for intelligently monitoring a specific indoor area according to claim 1, characterized in that: The optical flow algorithm is the LK optical flow algorithm.

6. The method for intelligently monitoring a specific indoor area according to claim 3, characterized in that: An implementation method of selecting a furniture category from all categories in a preset local area of ​​a target pixel based on the grayscale difference index includes: For each category in the preset local area of ​​the target pixel point, the mean value of the grayscale difference index of the pixel points of each category is recorded as the judgment index of the corresponding category, and the category corresponding to the smallest judgment index is selected as the furniture category of the target pixel point.

7. A video monitoring terminal, characterized in that: It comprises a processor, a memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; and the processor is used to call program instructions in the memory to execute the method described in any one of claims 1 to 6.

Images