Intelligent park management system based on Internet of Things

Abstract

The invention belongs to the technical field of data processing, and particularly relates to an intelligent park management system based on the Internet of Things, which comprises a data acquisition module, a data processing module and a target detection module, performing superpixel segmentation on each frame of image in the video data through a superpixel segmentation algorithm, and performing target detection and marking on a superpixel segmentation result of each frame of image in the video data through the trained classification neural network model; wherein the optical flow field of each frame of image is obtained through an optical flow method, the influence degree of motion blur on color features is determined according to optical flow vectors of seed points and to-be-segmented pixel points, and then three weights are determined; and weighting the color feature distance, the spatial position distance and the motion feature distance of the to-be-segmented pixel point and the seed point through the three weights, and constructing a distance measurement formula of the to-be-segmented pixel point and the seed point. According to the invention, the target detection precision of the video monitoring task in intelligent park management is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to an intelligent park management system based on the Internet of Things. Background Art

[0002] In smart park management, video surveillance system is one of the key technologies to achieve security monitoring, personnel management and resource optimization. With the expansion of park scale and the increase of management needs, traditional video surveillance system faces many challenges, such as huge data volume, high real-time requirements, and insufficient target detection accuracy.

[0003] In order to meet these challenges, the application of computer vision and deep learning technology in the field of video surveillance has gradually become a research hotspot in recent years. Among them, the combination of superpixel segmentation algorithm and classification neural network model provides an efficient and accurate solution for smart park management.

[0004] In the related technology, for example, the Chinese patent document with authorization announcement number CN109919223B discloses a target detection method and device based on a deep neural network, including: extracting deep features of different scales of video frames in the video to be tested based on a feature learning network; performing superpixel segmentation on the video frame to obtain a superpixel structure map; performing feature fusion on the deep features and the superpixel structure map to obtain a first fusion feature; obtaining spatial coding features based on the structure learning network and according to the first fusion feature; performing feature fusion on the deep features and the spatial coding features based on the feature fusion network to obtain a second fusion feature; using a conditional random field classifier to classify the second fusion feature, and performing bounding box regression on the target classification result to obtain the target detection result.

[0005] In video processing, each frame of an image is not just a static collection of pixels, but also contains motion information in the time dimension; for example, dynamic scenes such as vehicles driving and people walking will cause the pixels to change position between consecutive frames.

[0006] The superpixel segmentation algorithm mainly performs segmentation based on color features and spatial positions, ignoring the motion information of pixels in the video. This will lead to inaccurate segmentation results of moving targets (such as vehicles and people), and thus reduce the accuracy of target detection and recognition. Summary of the invention

[0007] In order to solve the technical problem that the superpixel segmentation algorithm ignores the motion information of the pixels in the video, resulting in inaccurate segmentation results of the moving target, and then causing the accuracy of target detection and recognition to decrease, the present invention provides an intelligent park management system based on the Internet of Things, the system includes the following modules: a data acquisition module, used to collect video data in the park; a data processing module, used to The superpixel segmentation algorithm performs superpixel segmentation on each frame of the video data to obtain the superpixel segmentation results of each frame of the image, wherein the method for obtaining the distance measurement formula between the pixel to be segmented and the seed point includes: obtaining the optical flow field of each frame of the image by the optical flow method, the optical flow field includes the optical flow vector of each pixel; determining the influence of motion blur on color features according to the optical flow vectors of the seed point and the pixel to be segmented, and determining three weights according to the influence of motion blur on color features; weighting the color feature distance, spatial position distance and motion feature distance between the pixel to be segmented and the seed point by the three weights, and constructing the distance measurement formula between the pixel to be segmented and the seed point; the target detection module is used to detect and mark the target of the superpixel segmentation result of each frame of the image in the video data through the trained classification neural network model.

[0008] The present invention introduces optical flow vectors to incorporate the motion information of pixels into the reference range of superpixel segmentation. In the distance measurement formula between the pixels to be segmented and the seed points, the motion feature distance between the pixels to be segmented and the seed points is added, thereby improving the accuracy and robustness of the superpixel segmentation results of the images in the video, making it more suitable for video surveillance tasks in intelligent park management and significantly improving the accuracy of target detection. In addition, the present invention determines three weights according to the influence of motion blur on color features, and can dynamically adjust the weights according to different motion states, so that the superpixel segmentation algorithm can adapt to various complex dynamic scenes in video surveillance, flexibly select more reliable features for segmentation, and enable superpixel segmentation to more accurately identify and segment targets when processing video surveillance tasks in intelligent park management.

[0009] Preferably, the optical flow vector of the pixel point is a vector , , are the velocity vectors of the optical flow along the X-axis and Y-axis respectively.

[0010] Preferably, the method for obtaining the influence of motion blur on color features includes: taking the mean of the color features of all pixel points within the influence range of the seed point as the representative color feature of the influence range of the seed point; calculating the Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point, and comparing the Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point with the compactness parameter The ratio of is used as the influence of motion blur on color features.

[0011] In dynamic scenes, motion blur can cause instability in color features. The present invention introduces the influence range of the seed point and characterizes the color features after motion blur by calculating the representative color features of the influence range. Then, the influence of motion blur on the color features is quantified by calculating the Euclidean distance between the color features of the seed point and the representative color features.

[0012] Preferably, the method for obtaining the influence range of the seed point comprises: for any seed point, taking the direction and modulus of the optical flow vector of the seed point as the optical flow direction and optical flow intensity of the seed point respectively; obtaining a vector whose direction is parallel to the optical flow direction of the seed point and whose spatial distance from the seed point is equal to The two straight lines of and straight line ; Obtain a straight line passing through the seed point and perpendicular to the optical flow direction of the seed point, recorded as straight line ; through the straight line The image is divided into two regions. In the region where the optical flow vector is located, the direction is perpendicular to the optical flow direction of the seed point and the distance from the spatial position of the seed point is equal to The straight line is denoted as ; The straight line , , as well as The area composed of is the influence range of the seed point. , Indicates taking the minimum value.

[0013] Preferably, determining the three weights according to the influence of motion blur on the color feature includes: recording the three weights as , and , the specific calculation formula is: ; ; ; In the formula, is the influence of motion blur on color features, is an exponential function with a natural constant as its base.

[0014] In dynamic scenes, motion blur can cause instability in color features. The present invention determines three weights according to the degree of influence of motion blur on color features, and can dynamically adjust the weights according to different motion states (such as stillness, slow motion, and fast motion). This allows the superpixel segmentation algorithm to adapt to various complex dynamic scenes in video surveillance and flexibly select more reliable features (color features or motion features) for segmentation. This allows superpixel segmentation to more accurately identify and segment targets when processing video surveillance tasks in smart park management.

[0015] Preferably, the distance measurement formula between the pixel to be segmented and the seed point is: ; In the formula, is the distance measurement value between the pixel to be segmented and the seed point, , , They are the color feature distance, spatial position distance and motion feature distance between the pixel to be segmented and the seed point, , and There are three weights respectively, is the spatial distance between adjacent seed points, and , , are the horizontal and vertical lengths of the image, respectively. , They are Compactness parameter and number hyperparameters in superpixel segmentation algorithms.

[0016] The present invention introduces optical flow vectors, incorporates the motion information of pixels into the reference range of superpixel segmentation, and adds the motion feature distance between the pixels to be segmented and the seed points in the distance measurement formula constructed, thereby improving the accuracy and robustness of the superpixel segmentation results of the image in the video. It is more suitable for video surveillance tasks in smart park management and can significantly improve the accuracy of target detection.

[0017] Preferably, the color feature distance between the pixel to be segmented and the seed point The calculation formula is: ; In the formula, is the color feature of the seed point, For the The color features of the pixels to be segmented, For lightness, For red and green indicators, It is a yellow-blue indicator.

[0018] Preferably, the spatial characteristic distance between the pixel to be segmented and the seed point The calculation formula is: ; In the formula, is the spatial position of the seed point, For the The spatial position of the pixels to be segmented, , are the horizontal and vertical axes respectively.

[0019] Preferably, the motion feature distance between the pixel to be segmented and the seed point The calculation formula is: ; In the formula, is the optical flow vector of the seed point, For the The optical flow vector of the pixels to be segmented, It means to find the magnitude of a vector.

[0020] When calculating the distance measurement value between the pixel to be segmented and the seed point, adding the similarity measurement of the optical flow vector between the pixel to be segmented and the seed point can ensure that the pixels in the superpixel area are not only similar in color features and spatial positions, but also maintain consistency in motion features.

[0021] Preferably, the superpixel segmentation results of each frame of the video data are used for target detection and labeling through a trained classification neural network model, including: inputting the superpixel segmentation results of each frame of the video data into a trained classification neural network model, outputting the type of each superpixel block in each frame of the image, wherein the types include personnel, vehicles and others, and quickly detecting the target object from real-time monitoring; performing connected domain analysis on superpixel blocks of the same type, grouping the superpixel blocks of the same type into multiple connected domains, and each connected domain serves as a target object of this type; for target objects of the same type, using bounding boxes of the same color to represent the position of target objects of this type.

[0022] The beneficial effects of the present invention are: The present invention introduces optical flow vectors to incorporate the motion information of pixels into the reference range of superpixel segmentation. In the distance measurement formula between the pixels to be segmented and the seed points, the motion feature distance between the pixels to be segmented and the seed points is added, thereby improving the accuracy and robustness of the superpixel segmentation results of the images in the video, making it more suitable for video surveillance tasks in intelligent park management and significantly improving the accuracy of target detection. In addition, the present invention determines three weights according to the influence of motion blur on color features, and can dynamically adjust the weights according to different motion states, so that the superpixel segmentation algorithm can adapt to various complex dynamic scenes in video surveillance, flexibly select more reliable features for segmentation, and enable superpixel segmentation to more accurately identify and segment targets when processing video surveillance tasks in intelligent park management. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a system block diagram schematically showing an intelligent park management system based on the Internet of Things in the present invention; Figure 2 is a schematic diagram schematically showing the influence range of a seed point. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0025] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0026] The embodiment of the present invention discloses an intelligent park management system based on the Internet of Things, referring to Figure 1 , including a data acquisition module 100 to a target detection module 300: The data acquisition module 100 is used to acquire video data within the park.

[0027] 1. Deploy high-definition cameras at key locations within the park to collect video data in real time. In addition to cameras, other sensors are also deployed to collect environmental data and assist in video surveillance. Key locations within the park include but are not limited to entrances and exits, parking lots, and areas around important facilities. Other sensors include but are not limited to infrared sensors, smoke sensors, and temperature and humidity sensors.

[0028] 2. Video data is collected in real time through cameras deployed in the park. The video data includes multiple frames of images, and the size of each frame is fixed. The size of the image is recorded as , , are the horizontal and vertical lengths of the image respectively; and the number of pixels in each frame is fixed, and the number of all pixels in each frame is equal to .

[0029] 3. Deploy an IoT gateway to collect video data collected by cameras and environmental data collected by sensors, and transmit the data to the local server through the network.

[0030] The data processing module 200 is used to obtain the superpixel segmentation result of each frame of the video data.

[0031] Superpixel segmentation is a technique that divides an image into multiple small, approximately uniform regions (superpixels); compared to traditional pixel-level segmentation, superpixel segmentation can significantly reduce the number of processing units while retaining key structural information of the image.

[0032] The superpixel segmentation algorithm is a simple, fast and easy-to-implement algorithm. The algorithm divides the image into multiple compact and uniform super-pixel regions by combining color features and spatial information; these super-pixel regions are not only similar in color but also continuous in space, making them very suitable for subsequent target detection tasks.

[0033] In smart park management, The superpixel segmentation algorithm can preprocess each frame of the video data and divide the image into multiple superpixel regions; this preprocessing step can effectively reduce the complexity of subsequent processing while improving the efficiency and accuracy of target detection.

[0034] The specific steps of the superpixel segmentation algorithm are as follows: (1) Obtain the color characteristics and spatial position of the pixel.

[0035] To ensure the segmentation accuracy, the superpixel segmentation algorithm needs to convert the image in RGB space into the image in CIELab space. For the image in CIELab space, the color feature of each pixel in the image is ,in, For lightness, For red and green indicators, It is a yellow-blue indicator.

[0036] At the same time, it is also necessary to obtain the spatial position of each pixel in the image. The specific method is: take the pixel in the lower left corner of each frame of the image as the origin, take the horizontal right direction at the origin as the positive direction of the X-axis, and take the vertical upward direction at the origin as the positive direction of the Y-axis to realize the construction of the rectangular coordinate system; obtain the spatial position of each pixel in each frame of the image in the rectangular coordinate system , Represents the horizontal coordinate of the pixel point, Represents the ordinate of the pixel point; since the horizontal length and vertical length of the image are , , so the horizontal coordinate of the pixel is The value range is , the vertical coordinate of the pixel The value range is .

[0037] (2) Set the seed point.

[0038] exist In the superpixel segmentation algorithm, the number of hyperparameters Determines the number of superpixel blocks in the segmentation result, so the number hyperparameter The specific value of can be set according to the actual application scenario and requirements. For smaller images, that is, images with a size less than 512×512, the number of hyperparameters The value range of is [10,100]; for medium-sized images, that is, images with sizes in the range of [512×512,1024×1024], the number hyperparameter The value range of is [100,500]; for larger images, that is, images larger than [1024×1024], the number of hyperparameters The value range is [500,1000].

[0039] Uniformly select from all pixels in each frame seed points, requiring the spatial distance between adjacent seed points equal , where the number of all pixels in each frame is equal to , , are the horizontal and vertical lengths of the image respectively.

[0040] When generating seed points, if the seed points fall on pixels belonging to edges or noise, that is, the seed points fall on pixels with larger gradients, the clustering effect will be affected. At this time, the seed points need to be relocated, where pixels with larger gradients refer to pixels with gradients greater than 30. The seed points are relocated by selecting the pixel with the smallest gradient in the 3×3 neighborhood of the initial seed point as the new seed point.

[0041] (3) Calculate the distance measurement value between the pixel to be segmented and the seed point, and assign the pixel to be segmented to the superpixel block corresponding to the seed point closest to it.

[0042] exist In the superpixel segmentation algorithm, only the distance measurement value between the pixel points within the search range of the seed point and the seed point is calculated to determine whether to divide the pixel point into the superpixel block corresponding to the seed point; therefore, for any seed point, a pixel point with the seed point as the center and a size equal to The area is used as the search range of the seed point; the pixel points to be segmented refer to the pixel points located in the search range of the seed point; the distance measurement value between each pixel point to be segmented and the seed point is calculated by the distance measurement formula between the pixel point to be segmented and the seed point.

[0043] (4) For each superpixel block obtained, recalculate the cluster center of each superpixel block, that is, the average value of the color features and spatial positions of all pixels in the superpixel block; repeat step (3) until the maximum number of iterations is reached, which is equal to 10.

[0044] (5) Post-processing all the generated superpixel blocks, including but not limited to removing small superpixel blocks and merging adjacent superpixel blocks, to optimize the segmentation results, and finally obtain the superpixel segmentation results of each frame image in the video data.

[0045] In video processing, each frame of an image is not just a static set of pixels, but also contains motion information in the time dimension; for example, dynamic scenes such as vehicles driving and people walking will cause the positions of pixels to change between consecutive frames; and the superpixel segmentation algorithm is mainly based on color features and spatial positions for segmentation, ignoring the motion information of pixels in the video, which will lead to inaccurate segmentation results of moving targets, and thus reduce the accuracy of target detection and recognition; therefore, the present invention obtains the optical flow field of each frame of the image through the optical flow method, determines the influence of motion blur on color features according to the optical flow vectors of the seed point and the pixel to be segmented, and then determines three weights, and weights the color feature distance, spatial position distance and motion feature distance between the pixel to be segmented and the seed point through the three weights, and constructs a distance measurement formula between the pixel to be segmented and the seed point, which is used to pass The superpixel segmentation algorithm performs superpixel segmentation on each frame of the video data to obtain the superpixel segmentation result of each frame of the image.

[0046] 1. Obtain the optical flow field of each frame image through the optical flow method.

[0047] Optical flow method is a technology used to estimate the pixel motion in an image sequence. It can capture the displacement of pixels between consecutive frames and express it in the form of an optical flow field. The optical flow field is a two-dimensional vector field that reflects the changing trend of each pixel on the image and can be regarded as the instantaneous velocity field generated by the movement of pixels with grayscale values ​​on the image plane.

[0048] The calculation of the optical flow field can be achieved through a variety of algorithms, the most common ones are sparse optical flow method and dense optical flow method; sparse optical flow method (such as LucasKanade algorithm) focuses on the key points in the image (usually corner points or significant feature points) and calculates the movement of these key points in consecutive frames; dense optical flow method (such as Farneback algorithm) calculates the movement of each pixel in the image to generate an optical flow field; LucasKanade algorithm and LucasKanade algorithm are both well-known technologies and will not be described here.

[0049] Specifically, for any frame of image, the Farneback algorithm is used to process the frame of image and its previous frame to obtain the optical flow field of the frame of image; the obtained optical flow field includes the optical flow vector of each pixel point, which can represent the movement of the pixel point between two adjacent frames of image. The optical flow vector is a vector. , , are the velocity vectors of the optical flow along the X-axis and Y-axis respectively.

[0050] 2. Determine the influence of motion blur on color features based on the optical flow vector of the seed point.

[0051] Specifically, for any seed point, the direction of the optical flow vector of the seed point is used as the optical flow direction of the seed point, and the modulus of the optical flow vector of the seed point is used as the optical flow intensity of the seed point.

[0052] Further, according to the optical flow direction and optical flow intensity of the seed point, the influence range of the seed point is obtained. The specific method is: obtain a direction that is parallel to the optical flow direction of the seed point and is equal to the spatial position distance of the seed point The two straight lines of and straight line ; Obtain a straight line passing through the seed point and perpendicular to the optical flow direction of the seed point, recorded as straight line ; through the straight line The image is divided into two regions. In the region where the optical flow vector is located, the direction is perpendicular to the optical flow direction of the seed point and the distance from the spatial position of the seed point is equal to The straight line is denoted as ; The straight line ,straight line ,straight line and straight lines The area composed of is taken as the influence range of the seed point, and the size of the influence range is equal to ,and , Indicates taking the minimum value.

[0053] For example, the schematic diagram of the influence range of the seed point is as follows: Figure 2 shown.

[0054] It should be noted that the optical flow direction and intensity of the seed point reflect the motion state of the seed point between consecutive frames. The greater the optical flow intensity, the more violent the movement of the seed point and the larger its influence range. The color features of all pixels within the influence range of the seed point can be used to evaluate the influence of motion blur on the color features.

[0055] Furthermore, the mean value of the color features of all pixels within the influence range of the seed point is taken as the representative color feature of the influence range of the seed point; it should be noted that this representative color feature can be regarded as the color feature after motion blur.

[0056] Furthermore, the Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point is calculated, and the Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point is added to the compactness parameter The ratio of is used as the influence of motion blur on color features. It should be noted that by calculating the Euclidean distance between the color features of the seed point and the representative color features, the influence of motion blur on color features can be quantified. The larger the distance, the more significant the influence of motion blur on color features.

[0057] Among them, the compactness parameter yes Well-known parameters in superpixel segmentation algorithms; compactness parameter The value of is [1,40]. The specific value can be set according to the actual application scenario and requirements. In this invention, the compactness parameter Set to 10.

[0058] The Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point represents the difference between the representative color feature of the influence range of the seed point and the color feature of the seed point.

[0059] 4. Determine three weights based on the impact of motion blur on color features.

[0060] It should be noted that, according to the influence of motion blur on color features, the three weights determined are finally used to construct the distance measurement formula between the pixel to be segmented and the seed point, and the three weights are used to weight the color feature distance, spatial position distance and motion feature distance in the distance measurement formula between the pixel to be segmented and the seed point, respectively. Among them, the greater the influence of motion blur on color features, the less the color feature distance between the pixel to be segmented and the seed point can reflect whether the pixel to be segmented and the seed point belong to the same object. At this time, it is more inclined to reflect whether the pixel to be segmented and the seed point belong to the same object through the motion feature distance between the pixel to be segmented and the seed point. Therefore, the smaller the first weight, the larger the third weight. Conversely, the smaller the influence of motion blur on color features, the more the color feature distance between the pixel to be segmented and the seed point can reflect whether the pixel to be segmented and the seed point belong to the same object. At this time, it is more inclined to reflect whether the pixel to be segmented and the seed point belong to the same object through the color feature distance between the pixel to be segmented and the seed point. Therefore, the larger the first weight, the smaller the third weight.

[0061] Specifically, according to the influence of motion blur on color features, three weights are determined and recorded as , and , the specific calculation formula is: ; ; ; In the formula, is the influence of motion blur on color features, is an exponential function with a natural constant as the base; the first weight The influence of motion blur on color characteristics is inversely proportional, and the third weight The influence of motion blur on color characteristics Therefore, the greater the influence of motion blur on color features, the smaller the first weight is, and the larger the third weight is.

[0062] in, , therefore, the sum of the three weights Always equal to 1.

[0063] It should be noted that the three weights are dynamically adjusted according to the influence of motion blur on color features; among them, the first weight is used to weight the color feature distance. When the influence of motion blur is large, the reliability of color features is reduced, so its weight is reduced; the second weight is used to weight the spatial position distance. The spatial position distance is usually relatively stable and is not affected by motion blur, so a relatively stable weight can be maintained; the third weight is used to weight the motion feature distance. When the influence of motion blur is large, the reliability of motion features is higher, so its weight is increased; this method of dynamically adjusting weights can ensure that under different motion states, the superpixel segmentation algorithm can more accurately reflect the similarities and differences between pixels.

[0064] For example, for stationary or slowly moving objects, color features may be more important, and in this case, motion blur has less impact, so the first weight is set larger; while for fast-moving objects, motion features are more reliable, and in this case, motion blur has a greater impact, so the third weight is set larger.

[0065] It should be further explained that in dynamic scenes, motion blur can lead to instability in the color features of pixels. The three weights used to weight the color feature distance, spatial position distance and motion feature distance between the pixel to be segmented and the seed point are determined by the degree of influence of motion blur on the color features. When subsequently constructing the distance measurement formula between the pixel to be segmented and the seed point, the color feature distance, spatial position distance and motion feature distance are weighted by these three weights. This allows dynamic adjustment of weights according to different motion states (such as still, slow motion, and fast motion), and flexible selection of more reliable features (color features or motion features) for superpixel segmentation. This allows the superpixel segmentation algorithm to adapt to various complex dynamic scenes, thereby improving the accuracy and robustness of the superpixel segmentation results of images in the video, and significantly improving the accuracy of target detection.

[0066] 5. According to the three weights, the color feature distance, spatial position distance and motion feature distance between the pixel to be segmented and the seed point are weighted, and a distance measurement formula between the pixel to be segmented and the seed point is constructed to calculate the distance measurement value between the pixel to be segmented and the seed point.

[0067] Specifically, the method for constructing the distance measurement formula between the pixel to be segmented and the seed point is: according to the three weights, the color feature distance, spatial position distance and motion feature distance between the pixel to be segmented and the seed point are weighted to obtain the distance measurement formula between the pixel to be segmented and the seed point. The distance measurement formula between the pixel to be segmented and the seed point is: ; In the formula, is the distance measurement value between the pixel to be segmented and the seed point, , , They are the color feature distance, spatial position distance and motion feature distance between the pixel to be segmented and the seed point, , and There are three weights respectively, is the compactness parameter, Well-known parameters in superpixel segmentation algorithms, is the spatial distance between adjacent seed points.

[0068] It should be noted that when performing superpixel segmentation on images in the video, the motion information of pixels belonging to the same moving target in the time series is considered to be similar. Therefore, the optical flow vector of the pixel is introduced to characterize the motion information. At the same time, the motion feature distance between the pixel to be segmented and the seed point is used to characterize the similarity measure of the optical flow vector of the pixel to be segmented and the seed point. Subsequently, in the distance measurement formula constructed between the pixel to be segmented and the seed point, the motion feature distance between the pixel to be segmented and the seed point is added, which can effectively improve the robustness and accuracy of superpixel segmentation in dynamic scenes such as video surveillance tasks in smart park management, and can significantly improve the accuracy of target detection.

[0069] Among them, the color feature distance between the pixel to be segmented and the seed point It is equal to the Euclidean distance between the color features of the pixel to be segmented and the seed point. The specific calculation formula is: ; In the formula, is the color feature of the seed point, For the The color features of the pixels to be segmented, For lightness, For red and green indicators, It is a yellow-blue indicator.

[0070] Among them, the spatial feature distance between the pixel to be segmented and the seed point It is equal to the Euclidean distance between the spatial features of the pixel to be segmented and the seed point. The specific calculation formula is: ; In the formula, is the spatial position of the seed point, For the The spatial position of the pixels to be segmented, , are the horizontal and vertical axes respectively.

[0071] Among them, the motion feature distance between the pixel to be segmented and the seed point The calculation formula is: ; In the formula, is the optical flow vector of the seed point, For the The optical flow vector of the pixels to be segmented, It means to find the magnitude of a vector.

[0072] The inner product of the optical flow vector of the pixel to be segmented and the seed point , and the modulus of the optical flow vector of the pixel to be segmented and the seed point , The ratio of the optical flow vector to obtain the cosine similarity of the angle between the pixel to be segmented and the seed point The larger the value, the smaller the angle of the optical flow vector, and the smaller the motion feature distance between the pixel to be segmented and the seed point; among them, the cosine similarity of the angle The value range is [-1,1], so the motion feature distance between the pixel to be segmented and the seed point The value range is [0,2].

[0073] It should be noted that when calculating the distance measurement value between the pixel to be segmented and the seed point, adding the similarity measurement of the optical flow vector between the pixel to be segmented and the seed point can ensure that the pixels in the superpixel area are not only similar in color features and spatial positions, but also maintain consistency in motion features.

[0074] The target detection module 300 is used to perform target detection and labeling on the superpixel segmentation results of each frame of the video data through the trained classification neural network model.

[0075] In smart campus management, video surveillance and target tracking are key technologies for achieving safe, efficient management and automated decision-making. Traditional monitoring systems mainly rely on manual monitoring, which is inefficient and error-prone. By introducing superpixel segmentation and neural network technology, automated target detection, tracking and behavior analysis can be achieved, improving monitoring efficiency and accuracy.

[0076] Specifically, in the process of smart park management, the superpixel segmentation results of each frame of the video data are input into the trained classification neural network model, and the type of each superpixel block in each frame of the image is output, including personnel, vehicles and others, so as to quickly detect the target object from real-time monitoring.

[0077] Furthermore, a connected domain analysis is performed on superpixel blocks of the same type, and the superpixel blocks of the same type are grouped into multiple connected domains, each of which is a target object of this type; for target objects of the same type, a bounding box of the same color is used to represent the position of the target object of this type.

[0078] The superpixel segmentation result of each frame image is taken as input, the type of each superpixel block is taken as output, and a neural network with an RNN network structure is trained through training samples as a classification neural network.

[0079] Among them, the training samples are historical video data collected and stored in the intelligent park management system. For each frame of the historical video data, the type label of each superpixel block in each frame is manually annotated; and if the type label is 1, it means that the type of the superpixel block is "personnel", the type label is 2, it means that the type of the superpixel block is "vehicle", and the type label is 3, it means that the type of the superpixel block is "other".

Claims

1. An intelligent park management system based on the Internet of Things, characterized in that: include: Data acquisition module, used to collect video data within the park; Data processing module, used to The superpixel segmentation algorithm performs superpixel segmentation on each frame of the video data to obtain the superpixel segmentation result of each frame of the image. The method for obtaining the distance measurement formula between the pixel point to be segmented and the seed point includes: The optical flow field of each frame image is obtained by the optical flow method, and the optical flow field includes the optical flow vector of each pixel point; according to the optical flow vectors of the seed point and the pixel point to be segmented, the influence of motion blur on the color feature is determined, and three weights are determined according to the influence of motion blur on the color feature; the color feature distance, spatial position distance and motion feature distance between the pixel point to be segmented and the seed point are weighted by the three weights, and a distance measurement formula between the pixel point to be segmented and the seed point is constructed; The target detection module is used to detect and mark the target of the superpixel segmentation results of each frame image in the video data through the trained classification neural network model.

2. According to the Internet of Things-based intelligent park management system of claim 1, it is characterized in that: The optical flow vector of the pixel is a vector , , are the velocity vectors of the optical flow along the X-axis and Y-axis respectively.

3. According to the Internet of Things-based intelligent park management system of claim 1, it is characterized in that: The method for obtaining the influence degree of motion blur on color features includes: The mean of the color features of all pixels within the influence range of the seed point is taken as the representative color feature of the influence range of the seed point; the Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point is calculated, and the Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point is added to the compactness parameter The ratio of is used as the influence of motion blur on color features.

4. The intelligent park management system based on the Internet of Things according to claim 3 is characterized in that: The method for obtaining the influence range of the seed point includes: For any seed point, the direction and modulus of the optical flow vector of the seed point are used as the optical flow direction and optical flow intensity of the seed point respectively; the direction is parallel to the optical flow direction of the seed point and the distance from the spatial position of the seed point is equal to The two straight lines of and straight line ; Obtain a straight line passing through the seed point and perpendicular to the optical flow direction of the seed point, recorded as straight line ; through the straight line The image is divided into two regions. In the region where the optical flow vector is located, the direction is perpendicular to the optical flow direction of the seed point and the distance from the spatial position of the seed point is equal to The straight line is denoted as ; The straight line , , as well as The area composed of is the influence range of the seed point. , Indicates taking the minimum value.

5. The intelligent park management system based on the Internet of Things according to claim 1 is characterized in that: According to the influence of motion blur on color features, three weights are determined, including: The three weights are respectively , and , the specific calculation formula is: ； ； ； In the formula, is the influence of motion blur on color features, is an exponential function with a natural constant as its base.

6. The intelligent park management system based on the Internet of Things according to claim 1 is characterized in that: The distance measurement formula between the pixel to be segmented and the seed point is: ； In the formula, is the distance measurement value between the pixel to be segmented and the seed point, , , They are the color feature distance, spatial position distance and motion feature distance between the pixel to be segmented and the seed point, , and There are three weights respectively, is the spatial distance between adjacent seed points, and , , are the horizontal and vertical lengths of the image, respectively. , They are Compactness parameter and number hyperparameters in superpixel segmentation algorithms.

7. The intelligent park management system based on the Internet of Things according to claim 6 is characterized in that: The color feature distance between the pixel to be segmented and the seed point The calculation formula is: ； In the formula, is the color feature of the seed point, For the The color features of the pixels to be segmented, For lightness, For red and green indicators, It is a yellow-blue indicator.

8. The intelligent park management system based on the Internet of Things according to claim 6 is characterized in that: The spatial characteristic distance between the pixel to be segmented and the seed point The calculation formula is: ； In the formula, is the spatial position of the seed point, For the The spatial position of the pixels to be segmented, , are the horizontal and vertical axes respectively.

9. The intelligent park management system based on the Internet of Things according to claim 6 is characterized in that: The motion feature distance between the pixel to be segmented and the seed point The calculation formula is: ； In the formula, is the optical flow vector of the seed point, For the The optical flow vector of the pixels to be segmented, It means to find the modulus of a vector.

10. The intelligent park management system based on the Internet of Things according to claim 1 is characterized in that: The trained classification neural network model is used to detect and mark the target on the superpixel segmentation result of each frame of the video data, including: The superpixel segmentation results of each frame of the video data are input into the trained classification neural network model, and the type of each superpixel block in each frame of the image is output, including people, vehicles and others, so as to quickly detect the target object from real-time monitoring; Connected domain analysis is performed on superpixel blocks of the same type, and superpixel blocks of the same type are grouped into multiple connected domains, each of which is a target object of this type; for target objects of the same type, the position of this type of target object is represented by a bounding box of the same color.

Images