An Internet of Things-based intelligent park management system

Abstract

The present invention belongs to the technical field of data processing, and particularly relates to an intelligent park management system based on the Internet of Things. The system includes a data acquisition module, a data processing module, and a target detection module. Superpixel segmentation is performed on each frame of image in the video data through a superpixel segmentation algorithm, and target detection and marking are performed on the superpixel segmentation results of each frame of image in the video data through a trained classification neural network model. Among them, the optical flow field of each frame of image is obtained through the optical flow method, and according to the optical flow vectors of the seed points and the pixels to be segmented, the influence degree of motion blur on the color features is determined, and then three weight values are determined. The color feature distance, spatial position distance, and motion feature distance between the pixels to be segmented and the seed points are weighted by the three weight values to construct a distance metric formula between the pixels to be segmented and the seed points. The present invention improves the target detection accuracy of video surveillance tasks in intelligent park management.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing. More specifically, the present invention relates to an intelligent park management system based on the Internet of Things. Background Art

[0002] In the management of intelligent parks, the video surveillance system is one of the key technologies for realizing security monitoring, personnel management, and resource optimization; with the expansion of the park scale and the improvement of management requirements, traditional video surveillance systems are facing many challenges, such as a large amount of data, high real-time requirements, insufficient target detection accuracy, etc.

[0003] In order to address these challenges, in recent years, the application of computer vision and deep learning technologies in the field of video surveillance has gradually become a research hotspot. Among them, the combination of the superpixel segmentation algorithm and the classification neural network model provides an efficient and accurate solution for intelligent park management.

[0004] In related technologies, for example, the Chinese patent document with the authorization announcement number CN109919223B discloses a target detection method and device based on a deep neural network, including: extracting deep features of different scales of video frames in a to-be-detected video based on a feature learning network; performing superpixel segmentation on the video frames to obtain a superpixel structure diagram; performing feature fusion on the deep features and the superpixel structure diagram to obtain a first fusion feature; obtaining a spatial coding feature based on a structure learning network and according to the first fusion feature; performing feature fusion on the deep features and the spatial coding feature based on a feature fusion network to obtain a second fusion feature; using a conditional random field classifier to perform target classification on the second fusion feature, and performing bounding box regression on the target classification result to obtain a target detection result.

[0005] In video processing, each frame of image is not only a static set of pixel points, but also contains motion information in the time dimension; for example, dynamic scenes such as vehicle driving and personnel walking will cause the pixel points to change positions between consecutive frames.

[0006] The superpixel segmentation algorithm mainly performs segmentation based on color features and spatial positions, ignoring the motion information of pixel points in the video, which will lead to inaccurate segmentation results of moving targets (such as vehicles and personnel), and further lead to a decrease in the accuracy of target detection and recognition. Summary of the Invention

[0007] To solve the technical problem that the above-mentioned superpixel segmentation algorithm ignores the motion information of pixel points in the video, resulting in inaccurate segmentation results of moving targets and further leading to a decrease in the accuracy of target detection and recognition, the present invention provides an intelligent park management system based on the Internet of Things. The system includes the following modules: a data acquisition module for collecting video data in the park; a data processing module for passing through The superpixel segmentation algorithm performs superpixel segmentation on each frame of the video data to obtain the superpixel segmentation result of each frame of the image. Among them, the method for obtaining the distance metric formula between the pixel to be segmented and the seed point includes: obtaining the optical flow field of each frame of the image through the optical flow method, where the optical flow field includes the optical flow vectors of each pixel point; determining the influence degree of motion blur on the color feature according to the optical flow vectors of the seed point and the pixel to be segmented, and determining three weight values according to the influence degree of motion blur on the color feature; constructing the distance metric formula between the pixel to be segmented and the seed point by weighting the color feature distance, spatial position distance, and motion feature distance between the pixel to be segmented and the seed point with the three weight values; the target detection module is used to perform target detection and marking on the superpixel segmentation result of each frame of the image in the video data through the trained classification neural network model.

[0008] In the present invention, by introducing the optical flow vector, the motion information of the pixel points is incorporated into the reference range of superpixel segmentation. In the constructed distance metric formula between the pixel to be segmented and the seed point, the motion feature distance between the pixel to be segmented and the seed point is increased, thereby improving the accuracy and robustness of the superpixel segmentation result of the images in the video, being more suitable for the video surveillance task in intelligent park management, and being able to significantly improve the accuracy of target detection; in addition, in the present invention, according to the influence degree of motion blur on the color feature, three weight values are determined, and the weight values can be dynamically adjusted according to different motion states, so that the superpixel segmentation algorithm can adapt to various complex dynamic scenes in video surveillance, flexibly select more reliable features for segmentation, and enable the superpixel segmentation to more accurately identify and segment the target when processing the video surveillance task in intelligent park management.

[0009] Preferably, the optical flow vector of the pixel point is a vector , , are respectively the velocity vectors of the optical flow along the X-axis and the Y-axis.

[0010] Preferably, the method for obtaining the influence degree of motion blur on the color feature includes: taking the mean value of the color features of all pixel points within the influence range of the seed point as the representative color feature of the influence range of the seed point; calculating the Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point, and taking the ratio of the Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point to the compactness parameter as the influence degree of motion blur on the color feature.

[0011] In a dynamic scene, motion blur can lead to instability of color features. The present invention introduces the influence range of seed points, and represents the color features after motion blur by calculating the representative color features of the influence range. Furthermore, the influence degree of motion blur on color features is quantified by calculating the Euclidean distance between the color features of the seed points and the representative color features.

[0012] Preferably, the method for obtaining the influence range of the seed points includes: for any one seed point, respectively taking the direction and magnitude of the optical flow vector of the seed point as the optical flow direction and optical flow intensity of the seed point; obtaining two straight lines whose directions are parallel to the optical flow direction of the seed point and whose spatial positions are equal to from the seed point, and respectively denoting them as straight line and straight line ; obtaining a straight line passing through the seed point and perpendicular to the optical flow direction of the seed point, and denoting it as straight line ; dividing the image into two regions by straight line , and in the region where the optical flow vector is located, obtaining a straight line whose direction is perpendicular to the optical flow direction of the seed point and whose spatial position is equal to from the seed point, and denoting it as straight line ; taking the region formed by straight line , , and as the influence range of the seed point, , indicating taking the minimum value.

[0013] Preferably, determining three weight values according to the influence degree of motion blur on color features includes: respectively denoting the three weight values as , and , and the specific calculation formula is: ; ; ; where is the influence degree of motion blur on color features, and is the exponential function with the natural constant as the base.

[0014] In a dynamic scene, motion blur can lead to instability of color features. According to the influence degree of motion blur on color features, the present invention determines three weight values, which can dynamically adjust the weight values according to different motion states (such as stationary, slow motion, fast motion), enabling the superpixel segmentation algorithm to adapt to various complex dynamic scenes in video surveillance, flexibly select more reliable features (color features or motion features) for segmentation, and enabling the superpixel segmentation to more accurately identify and segment targets when processing video surveillance tasks in intelligent park management.

[0015] Preferably, the distance metric formula between the pixel to be segmented and the seed point is: ; where is the distance metric value between the pixel to be segmented and the seed point, , , are respectively the color feature distance, the spatial position distance, and the motion feature distance between the pixel to be segmented and the seed point, , and are respectively three weight values, is the spatial distance between adjacent seed points, and , , are respectively the length of the image in the horizontal direction and the length in the vertical direction, , are respectively the compactness parameter and the number hyperparameter in the superpixel segmentation algorithm.

[0016] By introducing the optical flow vector, the present invention incorporates the motion information of pixel points into the reference range of superpixel segmentation. In the constructed distance metric formula between the pixel to be segmented and the seed point, the motion feature distance between the pixel to be segmented and the seed point is increased, thereby improving the accuracy and robustness of the superpixel segmentation result of the image in the video, being more applicable to the video surveillance task in intelligent park management, and being able to significantly improve the accuracy of target detection.

[0017] Preferably, the color feature distance between the pixel to be segmented and the seed point is calculated by the formula: ; where is the color feature of the seed point, is the color feature of the th pixel to be segmented, is the brightness, is the red-green index, is the yellow-blue index.

[0018] Preferably, the spatial feature distance between the pixel to be segmented and the seed point is calculated by the formula: ; where is the spatial position of the seed point, is the spatial position of the th pixel to be segmented, , are respectively the abscissa and the ordinate.

[0019] Preferably, the motion feature distance between the pixel to be segmented and the seed point is calculated by the formula: ; where, is the optical flow vector of the seed point, is the th optical flow vector of the pixel point to be segmented, represents calculating the modulus of the vector.

[0020] When calculating the distance metric value between the pixel point to be segmented and the seed point, adding the similarity metric of the optical flow vectors of the pixel point to be segmented and the seed point can ensure that the pixel points within the superpixel region are not only similar in color features and spatial positions but also consistent in motion features.

[0021] Preferably, using the trained classification neural network model to perform object detection and marking on the superpixel segmentation results of each frame of image in the video data includes: inputting the superpixel segmentation results of each frame of image in the video data into the trained classification neural network model, outputting the type of each superpixel block in each frame of image, where the type includes person, vehicle, and others, and quickly detecting the target object from real-time monitoring; performing connected component analysis on the superpixel blocks belonging to the same type, forming multiple connected components from the superpixel blocks belonging to the same type, and each connected component serves as a target object of this type; for the target objects belonging to the same type, using bounding boxes of the same color to represent the positions of the target objects of this type.

[0022] The beneficial effects of the present invention are as follows:

[0023] By introducing the optical flow vector, the present invention incorporates the motion information of pixel points into the reference range of superpixel segmentation. In the constructed distance metric formula between the pixel point to be segmented and the seed point, the motion feature distance between the pixel point to be segmented and the seed point is increased, thereby improving the accuracy and robustness of the superpixel segmentation results of images in the video, being more applicable to video surveillance tasks in intelligent park management, and being able to significantly improve the accuracy of object detection; in addition, according to the influence degree of motion blur on color features, the present invention determines three weight values, and the weight values can be dynamically adjusted according to different motion states, enabling the superpixel segmentation algorithm to adapt to various complex dynamic scenarios in video surveillance, flexibly select more reliable features for segmentation, and enabling the superpixel segmentation to more accurately identify and segment targets when dealing with video surveillance tasks in intelligent park management. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a schematic block diagram showing a smart park management system based on the Internet of Things in the present invention;

[0025] Figure 2 is a schematic diagram showing the influence range of the seed point. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0027] The following will describe in detail the specific implementation manners of the present invention with reference to the accompanying drawings.

[0028] An intelligent park management system based on the Internet of Things is disclosed in an embodiment of the present invention. Referring to Figure 1 , it includes a data acquisition module 100 to a target detection module 300:

[0029] The data acquisition module 100 is used to acquire video data in the park.

[0030] 1. High-definition cameras are deployed at key positions in the park for real-time acquisition of video data. In addition to cameras, other sensors are also deployed for acquiring environmental data to assist video monitoring. Key positions in the park include but are not limited to entrances and exits, parking lots, and areas around important facilities. Other sensors include but are not limited to infrared sensors, smoke sensors, and temperature and humidity sensors.

[0031] 2. Video data is acquired in real time through the cameras deployed in the park. The video data includes multiple frames of images, and the size of each frame of image is fixed and unchanged. Denote the size of the image as , , are the length of the image in the horizontal direction and the length in the vertical direction respectively; and the number of pixel points in each frame of image is fixed and unchanged, and the number of all pixel points in each frame of image is equal to .

[0032] 3. An Internet of Things gateway is deployed to collect the video data acquired by the cameras and the environmental data acquired by the sensors, and transmit the data to the local server through the network.

[0033] The data processing module 200 is used to obtain the superpixel segmentation result of each frame of image in the video data.

[0034] Superpixel segmentation is a technology that divides an image into multiple small, approximately uniform regions (superpixels). Compared with traditional pixel-level segmentation, superpixel segmentation can significantly reduce the number of processing units while retaining the key structural information of the image.

[0035] The superpixel segmentation algorithm is an algorithm with simple idea, fast speed, and convenient implementation. The algorithm combines color features and spatial information to segment the image into multiple compact and uniform superpixel regions; these superpixel regions not only have color similarity but also maintain spatial continuity, making them very suitable for subsequent object detection tasks.

[0036] In the management of smart campuses, The superpixel segmentation algorithm can preprocess each frame of the video data, segmenting the image into multiple superpixel regions; this preprocessing step can effectively reduce the complexity of subsequent processing while improving the efficiency and accuracy of object detection.

[0037] The specific steps of the superpixel segmentation algorithm are as follows:

[0038] (1) Obtain the color features and spatial positions of the pixel points.

[0039] To ensure the segmentation accuracy, the superpixel segmentation algorithm first converts the image in the RGB space into an image in the CIELab space. For the image in the CIELab space, the color feature of each pixel point in the image is , where is the lightness, is the red-green index, is the yellow-blue index.

[0040] Meanwhile, it is also necessary to obtain the spatial position of each pixel point in the image. The specific method is as follows: Taking the pixel point at the lower left corner of each frame of the image as the origin, the horizontal right direction at the origin is taken as the positive direction of the X-axis, and the vertical upward direction at the origin is taken as the positive direction of the Y-axis to construct a rectangular coordinate system; obtain the spatial position of each pixel point in each frame of the image in the rectangular coordinate system , represents the abscissa of the pixel point, represents the ordinate of the pixel point; since the length of the image in the horizontal direction and the length in the vertical direction are , respectively, therefore, the abscissa of the pixel point has a value range of , and the ordinate of the pixel point has a value range of .

[0041] (2) Set the seed points.

[0042] In the superpixel segmentation algorithm, the number hyperparameter determines the number of superpixel blocks in the segmentation result. Therefore, the number hyperparameter The specific value can be set according to the actual application scenario and requirements. For a smaller image, that is, an image with a size smaller than 512×512, the quantity hyperparameter ranges from [10, 100]; for a medium-sized image, that is, an image with a size in the range of [512×512, 1024×1024], the quantity hyperparameter ranges from [100, 500]; for a larger image, that is, an image with a size larger than [1024×1024], the quantity hyperparameter ranges from [500, 1000].

[0043] Uniformly select seed points among all the pixel points of each frame of the image. It is required that the spatial distance between adjacent seed points is equal to , where the number of all pixel points in each frame of the image is equal to , , are the length of the image in the horizontal direction and the length in the vertical direction respectively.

[0044] When generating the seed points, if the seed points fall on the pixel points belonging to the edge or noise, that is, the seed points fall on the pixel points with a large gradient, it will affect the clustering effect. At this time, the seed points need to be repositioned. Among them, the pixel points with a large gradient refer to the pixel points with a gradient greater than 30; by selecting the pixel point with the smallest gradient within the 3×3 neighborhood of the initial seed point as the new seed point, the repositioning of the seed points is realized.

[0045] (3) Calculate the distance metric value between the pixel points to be segmented and this seed point, and assign the pixel points to be segmented to the superpixel block corresponding to the nearest seed point.

[0046] In the superpixel segmentation algorithm, only calculate the distance metric value between the pixel points within the search range of the seed point and this seed point, and then judge whether to divide the pixel points into the superpixel block corresponding to this seed point; therefore, for any one seed point, take the area centered on this seed point and with a size equal to as the search range of this seed point; then the pixel points to be segmented refer to the pixel points located within the search range of this seed point; calculate the distance metric value between each pixel point to be segmented and this seed point through the distance metric formula of the pixel points to be segmented and the seed point.

[0047] (4) For each obtained superpixel block, recalculate the clustering center of each superpixel block, that is, the average value of the color features and spatial positions of all pixel points within the superpixel block; repeat step (3) until the maximum number of iterations is reached, and the maximum number of iterations is equal to 10.

[0048] (5) Post-process all the generated superpixel blocks, including but not limited to removing small superpixel blocks and merging adjacent superpixel blocks, to optimize the segmentation result, and finally obtain the superpixel segmentation result of each frame image in the video data.

[0049] In video processing, each frame image is not only a set of static pixel points, but also contains motion information in the time dimension; for example, dynamic scenes such as vehicle driving and people walking will cause the positions of pixel points to change between consecutive frames; while the superpixel segmentation algorithm mainly segments based on color features and spatial positions, ignoring the motion information of pixel points in the video, which will lead to inaccurate segmentation results of moving objects, and further lead to a decrease in the accuracy of object detection and recognition; therefore, in the present invention, an optical flow field of each frame image is obtained through the optical flow method, and according to the optical flow vectors of the seed points and the pixels to be segmented, the influence degree of motion blur on color features is determined, and then three weight values are determined. The color feature distance, spatial position distance, and motion feature distance between the pixels to be segmented and the seed points are weighted by the three weight values to construct a distance metric formula for the pixels to be segmented and the seed points, which is used to perform superpixel segmentation on each frame image in the video data through the superpixel segmentation algorithm to obtain the superpixel segmentation result of each frame image.

[0050] 1. Obtain the optical flow field of each frame image through the optical flow method.

[0051] The optical flow method is a technique for estimating the motion of pixels in an image sequence, which can capture the displacement of pixel points between consecutive frames and represent it in the form of an optical flow field; the optical flow field is a two-dimensional vector field, which reflects the change trend of each pixel point on the image and can be regarded as an instantaneous velocity field generated by the motion of pixel points with gray values on the image plane.

[0052] The calculation of the optical flow field can be realized through various algorithms. Common ones include sparse optical flow method and dense optical flow method; the sparse optical flow method (such as the LucasKanade algorithm) focuses on the key points in the image (usually corner points or significant feature points) and calculates the motion of these key points in consecutive frames; the dense optical flow method (such as the Farneback algorithm) calculates the motion of each pixel point in the image to generate an optical flow field; both the LucasKanade algorithm and the LucasKanade algorithm are well-known techniques and will not be elaborated here.

[0053] Specifically, for any frame image, through the Farneback algorithm, the frame image and its previous frame image are processed to obtain the optical flow field of the frame image; the obtained optical flow field includes the optical flow vectors of each pixel point, and the optical flow vector can represent the movement of the pixel point between two adjacent frame images. The optical flow vector is a vector , , are the velocity vectors of the optical flow along the X-axis and Y-axis respectively.

[0054] 2. Determine the influence of motion blur on color features based on the optical flow vector of the seed point.

[0055] Specifically, for any seed point, the direction of the optical flow vector of the seed point is used as the optical flow direction of the seed point, and the modulus of the optical flow vector of the seed point is used as the optical flow intensity of the seed point.

[0056] Further, according to the optical flow direction and optical flow intensity of the seed point, the influence range of the seed point is obtained. The specific method is: obtain a direction that is parallel to the optical flow direction of the seed point and is equal to the spatial position distance of the seed point The two straight lines of and straight line ; Obtain a straight line passing through the seed point and perpendicular to the optical flow direction of the seed point, recorded as straight line ; through the straight line The image is divided into two regions. In the region where the optical flow vector is located, the direction is perpendicular to the optical flow direction of the seed point and the distance from the spatial position of the seed point is equal to The straight line is denoted as ; The straight line ,straight line ,straight line and straight lines The area composed of is taken as the influence range of the seed point, and the size of the influence range is equal to ,and , Indicates taking the minimum value.

[0057] For example, the schematic diagram of the influence range of the seed point is as follows: Figure 2 shown.

[0058] It should be noted that the optical flow direction and intensity of the seed point reflect the motion state of the seed point between consecutive frames. The greater the optical flow intensity, the more violent the movement of the seed point and the larger its influence range. The color features of all pixels within the influence range of the seed point can be used to evaluate the influence of motion blur on the color features.

[0059] Furthermore, the mean value of the color features of all pixels within the influence range of the seed point is taken as the representative color feature of the influence range of the seed point; it should be noted that this representative color feature can be regarded as the color feature after motion blur.

[0060] Further, calculate the Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point, and take the ratio of the Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point to the compactness parameter as the influence degree of motion blur on the color feature; it should be noted that by calculating the Euclidean distance between the color feature of the seed point and the representative color feature, the influence degree of motion blur on the color feature can be quantified. The larger the distance, the more significant the influence of motion blur on the color feature.

[0061] Among them, the compactness parameter is a well-known parameter in the superpixel segmentation algorithm; the value-taking method of the compactness parameter is [1, 40], and the specific value can be set according to the actual application scenario and requirements. In the present invention, the compactness parameter is set to 10.

[0062] Among them, the Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point represents the difference between the representative color feature of the influence range of the seed point and the color feature of the seed point.

[0063] 4. Determine three weight values according to the influence degree of motion blur on the color feature.

[0064] It should be noted that according to the influence degree of motion blur on the color feature, the three determined weight values are finally used to construct the distance metric formula between the pixel point to be segmented and the seed point, and the three weight values are respectively used to weight the color feature distance, spatial position distance, and motion feature distance in the distance metric formula between the pixel point to be segmented and the seed point. Among them, the greater the influence degree of motion blur on the color feature, the less the color feature distance between the pixel point to be segmented and the seed point can reflect whether the pixel point to be segmented and the seed point belong to the same object. At this time, it is more inclined to reflect whether the pixel point to be segmented and the seed point belong to the same object through the motion feature distance between the pixel point to be segmented and the seed point. Therefore, the first weight value is smaller, and the third weight value is larger. On the contrary, the smaller the influence degree of motion blur on the color feature, the more the color feature distance between the pixel point to be segmented and the seed point can reflect whether the pixel point to be segmented and the seed point belong to the same object. At this time, it is more inclined to reflect whether the pixel point to be segmented and the seed point belong to the same object through the color feature distance between the pixel point to be segmented and the seed point. Therefore, the first weight value is larger, and the third weight value is smaller.

[0065] Specifically, according to the influence degree of motion blur on the color feature, determine three weight values, and denote the three weight values as 、 and , and the specific calculation formula is:

[0066] ;

[0067] ;

[0068] ;

[0069] In the formula, is the influence degree of motion blur on color features, is an exponential function with the natural constant as the base; the first weight is inversely proportional to the influence degree of motion blur on color features , and the third weight is directly proportional to the influence degree of motion blur on color features . Therefore, the greater the influence degree of motion blur on color features, the smaller the first weight, and the greater the third weight.

[0070] Among them, , so the sum of the three weights is always equal to 1.

[0071] It should be noted that according to the influence degree of motion blur on color features, the three weights are dynamically adjusted; among them, the first weight is used to weight the color feature distance. When the motion blur has a greater impact, the reliability of the color features decreases, so its weight is reduced; the second weight is used to weight the spatial position distance. The spatial position distance is usually relatively stable and not affected by motion blur, so a relatively stable weight can be maintained; the third weight is used to weight the motion feature distance. When the motion blur has a greater impact, the reliability of the motion features is higher, so its weight is increased; this method of dynamically adjusting the weights can ensure that in different motion states, the superpixel segmentation algorithm can more accurately reflect the similarity and difference between pixel points.

[0072] For example, for a stationary or slowly moving object, color features may be more important. At this time, the motion blur has a smaller impact. Therefore, the first weight is set to be larger; while for a fast moving object, the motion features are more reliable. At this time, the motion blur has a greater impact. Therefore, the third weight is set to be larger.

[0073] Furthermore, it should be noted that in a dynamic scene, motion blur will cause instability in the color features of pixel points. Based on the degree of influence of motion blur on color features, three weights are determined for weighting the color feature distance, spatial position distance, and motion feature distance between the pixel points to be segmented and the seed points. When constructing the distance metric formula for the pixel points to be segmented and the seed points subsequently, these three weights are used to weight the color feature distance, spatial position distance, and motion feature distance, thereby achieving dynamic adjustment of the weights according to different motion states (such as stationary, slow motion, and fast motion), flexibly selecting more reliable features (color features or motion features) for superpixel segmentation, enabling the superpixel segmentation algorithm to adapt to various complex dynamic scenes, thus improving the accuracy and robustness of the superpixel segmentation results of images in the video, and significantly improving the accuracy of object detection.

[0074] 5. According to the three weights, weight the color feature distance, spatial position distance, and motion feature distance between the pixel points to be segmented and the seed points, and construct a distance metric formula for the pixel points to be segmented and the seed points to calculate the distance metric value between the pixel points to be segmented and the seed points.

[0075] Specifically, the method for constructing the distance metric formula for the pixel points to be segmented and the seed points is as follows: According to the three weights, weight the color feature distance, spatial position distance, and motion feature distance between the pixel points to be segmented and the seed points to obtain the distance metric formula for the pixel points to be segmented and the seed points. Then the constructed distance metric formula for the pixel points to be segmented and the seed points is:

[0076] ;

[0077] In the formula, is the distance metric value between the pixel points to be segmented and the seed points, , , are respectively the color feature distance, spatial position distance, and motion feature distance between the pixel points to be segmented and the seed points, , and are respectively the three weights, is a compactness parameter, which is a well-known parameter in the superpixel segmentation algorithm, is the spatial distance between adjacent seed points.

[0078] It should be noted that when performing superpixel segmentation on the images in the video, considering that the motion information of the pixel points belonging to the same moving object is similar in the time series, therefore, the optical flow vector of the pixel points is introduced to characterize the motion information. At the same time, the motion feature distance between the pixel point to be segmented and the seed point is used to characterize the similarity measure of the optical flow vectors of the pixel point to be segmented and the seed point. Subsequently, by adding the motion feature distance between the pixel point to be segmented and the seed point in the constructed distance metric formula for the pixel point to be segmented and the seed point, the robustness and accuracy of superpixel segmentation in dynamic scenarios such as video surveillance tasks in intelligent park management can be effectively improved, and the accuracy of object detection can be significantly improved.

[0079] Among them, the color feature distance between the pixel point to be segmented and the seed point is equal to the Euclidean distance of the color features of the pixel point to be segmented and the seed point, and the specific calculation formula is:

[0080] ;

[0081] In the formula, is the color feature of the seed point, is the th color feature of the pixel point to be segmented, is the brightness, is the red-green index, is the yellow-blue index.

[0082] Among them, the spatial feature distance between the pixel point to be segmented and the seed point is equal to the Euclidean distance of the spatial features of the pixel point to be segmented and the seed point, and the specific calculation formula is:

[0083] ;

[0084] In the formula, is the spatial position of the seed point, is the th spatial position of the pixel point to be segmented, , are the abscissa and ordinate respectively.

[0085] Among them, the calculation formula of the motion feature distance between the pixel point to be segmented and the seed point is:

[0086] ;

[0087] In the formula, is the optical flow vector of the seed point, is the th optical flow vector of the pixel point to be segmented, represents calculating the modulus of the vector.

[0088] Through the inner product of the optical flow vector between the pixel to be segmented and the seed point , and the magnitude of the optical flow vector between the pixel to be segmented and the seed point , The cosine similarity of the included angle of the optical flow vector between the pixel to be segmented and the seed point is obtained by taking the ratio of . The larger this value is, the smaller the included angle of the optical flow vector is, and the smaller the motion feature distance between the pixel to be segmented and the seed point is. Among them, the cosine similarity of the included angle ranges from [-1, 1]. Therefore, the motion feature distance between the pixel to be segmented and the seed point ranges from [0, 2].

[0089] It should be noted that when calculating the distance metric value between the pixel to be segmented and the seed point, adding a similarity metric of the optical flow vector between the pixel to be segmented and the seed point can ensure that the pixels within the superpixel region are not only similar in color features and spatial positions but also consistent in motion features.

[0090] The target detection module 300 is used to perform target detection and marking on the superpixel segmentation results of each frame of image in the video data through a trained classification neural network model.

[0091] In the intelligent park management, video surveillance and target tracking are key technologies for achieving safe, efficient management, and automated decision-making. Traditional surveillance systems mainly rely on manual monitoring, which is inefficient and error-prone. By introducing superpixel segmentation and neural network technologies, automated target detection, tracking, and behavior analysis can be achieved, improving the surveillance efficiency and accuracy.

[0092] Specifically, during the intelligent park management process, the superpixel segmentation results of each frame of image in the video data are input into a trained classification neural network model, and the type of each superpixel block in each frame of image is output. The types include people, vehicles, and others, and target objects are quickly detected from real-time monitoring.

[0093] Furthermore, connected component analysis is performed on the superpixel blocks belonging to the same type, and the superpixel blocks belonging to the same type are grouped into multiple connected components. Each connected component serves as a target object of this type. For the target objects belonging to the same type, the position of the target object of this type is represented by a bounding box of the same color.

[0094] Taking the superpixel segmentation results of each frame of image as the input and the type of each superpixel block as the output, a neural network with an RNN network structure is trained through training samples to serve as the classification neural network.

[0095] Among them, the training samples are historical video data collected and stored in the intelligent park management system. For each frame of image in the historical video data, the type labels of each superpixel block in each frame of image are manually marked; and if the type label is 1, it means that the type of the superpixel block is "person", if the type label is 2, it means that the type of the superpixel block is "vehicle", and if the type label is 3, it means that the type of the superpixel block is "others".

Claims

1. An intelligent park management system based on the Internet of Things, characterized in that: include: Data acquisition module, used to collect video data within the park; Data processing module, used to The superpixel segmentation algorithm performs superpixel segmentation on each frame of the video data to obtain the superpixel segmentation result of each frame of the image. The method for obtaining the distance measurement formula between the pixel point to be segmented and the seed point includes: The optical flow field of each frame image is obtained by the optical flow method, and the optical flow field includes the optical flow vector of each pixel point; according to the optical flow vectors of the seed point and the pixel point to be segmented, the influence of motion blur on the color feature is determined, and three weights are determined according to the influence of motion blur on the color feature; the color feature distance, spatial position distance and motion feature distance between the pixel point to be segmented and the seed point are weighted by the three weights, and a distance measurement formula between the pixel point to be segmented and the seed point is constructed; The method of determining three weights according to the influence of motion blur on color features includes: recording the three weights as , and , the specific calculation formula is: ; ; ; In the formula, is the influence of motion blur on color features, is an exponential function with a natural constant as base; The target detection module is used to detect and mark the target of the superpixel segmentation results of each frame image in the video data through the trained classification neural network model.

2. According to the Internet of Things-based intelligent park management system of claim 1, it is characterized in that: The optical flow vector of the pixel is a vector , , are the velocity vectors of the optical flow along the X-axis and Y-axis respectively.

3. According to the Internet of Things-based intelligent park management system of claim 1, it is characterized in that: The method for obtaining the influence degree of motion blur on color features includes: The mean of the color features of all pixels within the influence range of the seed point is taken as the representative color feature of the influence range of the seed point; the Euclidean distance between the representative color feature of the influence range of the seed point and the color feature of the seed point is calculated, and the Euclidean distance is added to the compactness parameter The ratio of is used as the influence of motion blur on color features.

4. The intelligent park management system based on the Internet of Things according to claim 3 is characterized in that: The method for obtaining the influence range of the seed point includes: For any seed point, the direction and modulus of the optical flow vector of the seed point are used as the optical flow direction and optical flow intensity of the seed point respectively; the direction is parallel to the optical flow direction of the seed point and the distance from the spatial position of the seed point is equal to The two straight lines of and straight line ; Obtain a straight line passing through the seed point and perpendicular to the optical flow direction of the seed point, recorded as straight line ; through the straight line The image is divided into two regions. In the region where the optical flow vector is located, the direction is perpendicular to the optical flow direction of the seed point and the distance from the spatial position of the seed point is equal to The straight line is denoted as ; The straight line , , as well as The area composed of is the influence range of the seed point. , Indicates taking the minimum value.

5. The intelligent park management system based on the Internet of Things according to claim 1 is characterized in that: The distance measurement formula between the pixel to be segmented and the seed point is: ； In the formula, is the distance measurement value between the pixel to be segmented and the seed point, , , They are the color feature distance, spatial position distance and motion feature distance between the pixel to be segmented and the seed point, , and There are three weights respectively, is the spatial distance between adjacent seed points, and , , are the horizontal and vertical lengths of the image, respectively. , They are Compactness parameter and number hyperparameters in superpixel segmentation algorithms.

6. The intelligent park management system based on the Internet of Things according to claim 5 is characterized in that: The color feature distance between the pixel to be segmented and the seed point The calculation formula is: ； In the formula, is the color feature of the seed point, For the The color features of the pixels to be segmented, For lightness, For red and green indicators, It is a yellow-blue indicator.

7. The intelligent park management system based on the Internet of Things according to claim 5 is characterized in that: The spatial characteristic distance between the pixel to be segmented and the seed point The calculation formula is: ； In the formula, is the spatial position of the seed point, For the The spatial position of the pixels to be segmented, , are the horizontal and vertical axes respectively.

8. The intelligent park management system based on the Internet of Things according to claim 5 is characterized in that: The motion feature distance between the pixel to be segmented and the seed point The calculation formula is: ； In the formula, is the optical flow vector of the seed point, For the The optical flow vector of the pixels to be segmented, It means to find the magnitude of a vector.

9. The intelligent park management system based on the Internet of Things according to claim 1 is characterized in that: The trained classification neural network model is used to detect and mark the target on the superpixel segmentation result of each frame of the video data, including: The superpixel segmentation results of each frame of the video data are input into the trained classification neural network model, and the type of each superpixel block in each frame of the image is output, including people, vehicles and others, so as to quickly detect the target object from real-time monitoring; Connected domain analysis is performed on superpixel blocks of the same type, and superpixel blocks of the same type are grouped into multiple connected domains, each of which is a target object of this type; for target objects of the same type, the position of this type of target object is represented by a bounding box of the same color.

Images