Channel occupation monitoring method and system

Abstract

The invention discloses a channel occupation monitoring method and system, and relates to the technical field of safety monitoring, and the method comprises the steps: collecting and preprocessing initial multi-dimensional data of a fire fighting channel, and mapping the multi-dimensional data to a space coordinate system to generate an initial three-dimensional model; and comparing the real-time multi-dimensional data with the data in the initial three-dimensional model to identify a changing occupying object and generate a preliminary bounding box, and optimizing the detected boundary of the occupying object. According to the method, the multi-dimensional data in the fire fighting access is collected and mapped to the space coordinate system to generate the initial three-dimensional model, the object bounding box is generated based on the change of the three-dimensional model and optimized, so that the rotation information of the object can be captured, the sudden change point of the boundary is identified, and the discontinuous area is corrected; therefore, the bounding box which better accords with the real shape of the object is generated, the optimized bounding box reflects the accurate contour of the object, the recognition precision and stability of the system in a complex scene are improved, and the false alarm rate and delay are effectively reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of safety monitoring, and in particular to a channel occupancy monitoring method and system. Background Art

[0002] In recent years, with the continuous advancement of intelligent technology, channel monitoring technology has been widely used in various security and management application scenarios. In the fields of traffic management, personnel evacuation, fire emergency, etc., monitoring channel occupancy is crucial to ensure the smooth flow of channels. Traditional channel occupancy monitoring methods mainly rely on camera monitoring, which identifies occupied objects through simple two-dimensional images. However, two-dimensional monitoring technology has obvious limitations. For example, it is difficult to accurately capture the three-dimensional form of objects and their occupied space due to the viewing angle, lighting conditions, and occlusion problems in complex scenes. In addition, traditional monitoring technologies often rely on manual intervention and have a low degree of automation, which makes it difficult for real-time and response efficiency to meet complex application requirements. Summary of the invention

[0003] In view of the problems existing in the above-mentioned existing channel occupancy monitoring methods and systems, the present invention is proposed.

[0004] Therefore, the problem to be solved by the present invention is that two-dimensional monitoring technology is difficult to accurately capture the three-dimensional shape of an object and its occupied space. Traditional monitoring technology often relies on manual intervention and has a low degree of automation, resulting in real-time and response efficiency that are difficult to meet complex application requirements.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: a channel occupancy monitoring method, which includes collecting initial multidimensional data of the fire channel and performing preprocessing, mapping the multidimensional data to a spatial coordinate system to generate an initial three-dimensional model; comparing the real-time multidimensional data with the data in the initial three-dimensional model to identify the changed occupied objects and generate a preliminary bounding box, and optimizing the detected occupied object boundaries; calculating the volume of the occupied object according to the boundary detection results, and taking different control measures according to the volume calculation results.

[0006] As a preferred solution of the channel occupancy monitoring method described in the present invention, the collection of initial multi-dimensional data of the fire channel and pre-processing thereof refers to deploying RGB cameras, lidars and depth sensors in the fire channel, collecting the color information, two-dimensional contours, three-dimensional point cloud data and depth maps of the initial objects in the fire channel, and pre-processing and time synchronization calibration of the collected multi-dimensional data.

[0007] As a preferred solution of the channel occupancy monitoring method of the present invention, wherein: the multi-dimensional data is mapped into a spatial coordinate system to generate an initial three-dimensional model, the depth map and the point cloud data are mapped into a unified three-dimensional coordinate system, and the mapped depth map and point cloud data are combined to form a three-dimensional geometric shape of the object;

[0008] The color and two-dimensional boundary of each object are extracted through the two-dimensional contours of the objects collected in real time. The two-dimensional contours generated by the RGB image and the depth map are combined to generate an initial three-dimensional model of the fire passage. After standardization, the three-dimensional model is used as the standard state of the fire passage without occupied objects.

[0009] As a preferred solution of the channel occupancy monitoring method of the present invention, wherein: the comparison of real-time multi-dimensional data and data in the initial three-dimensional model to identify the changed occupied objects and generate a preliminary bounding box refers to collecting real-time multi-dimensional data in the fire channel and pre-processing the real-time collected data and the initial three-dimensional model frame by frame, calculating the difference between the position, color and depth of each point and the corresponding point in the initial three-dimensional model through the ICP algorithm, comparing all points one by one, weighted fusion of the obtained difference values ​​to obtain a comprehensive change index, setting a change threshold K, and marking the area with a change index greater than or equal to K as a change area;

[0010] The Kmeans++ clustering algorithm is used to cluster the change points in the marked change area. The clustering data k is set according to the change area detected in real time. All marked change points are assigned to different clustering categories through iterative optimization. For each clustering result, a preliminary bounding box of the occupied object is generated.

[0011] As a preferred embodiment of the channel occupancy monitoring method of the present invention, wherein: the optimization of the detected occupied object boundary refers to detecting the object in the preliminary boundary box and calculating the rotation angle of each vertex of the object boundary. Identify the rotation information of an object:

[0012]

[0013] In the formula, is the distance between two boundary points in the y-axis direction, is the distance between two boundary points in the x-axis direction, and is the velocity of the object in the x and y directions, and is the acceleration of the object in the x and y directions, is the exponential decay coefficient, I(a) is the rotational inertia of the object, a is the angular velocity of the object, r is the rotation radius representing the distance of the object relative to the rotation axis, () is the inverse tangent function;

[0014] Mark the angular position of the rotating object based on the calculated angle information of the object boundary, and calculate the angle change of each boundary vertex Perform calculations to identify angle mutation points:

[0015]

[0016] Where K(t-τ) is the Gaussian convolution kernel, is the angle of the i-th boundary point at time t, is the angle of the i+1th boundary point at time t, is the time derivative of the angle of the ith boundary point, and is the boundary of the time interval, dτ is the small change in the integral variable τ;

[0017] Analyze the angle change of each mutation point. When the angle mutations of multiple adjacent vertices occur in the same area, there is discontinuity at the boundary of the area. Record each confirmed mutation point and discontinuous area.

[0018] The rotation angle of each mutation point is encoded using the complex exponential encoding formula, and the angle information of the object boundary is converted into a smooth expression in the complex domain;

[0019] The smoothness of the coding is controlled by adjusting the angular frequency in real time according to the boundary characteristics of the object;

[0020] Recalculate and decode the rotation angle information through the reverse decoding formula of the complex exponential encoding;

[0021] Reconstruct the boundary segments according to the rotation angle of each vertex and the coordinates of the adjacent vertices;

[0022] After smoothness detection of the newly generated boundary segments, all boundary segments are reassembled to generate a complete object bounding box;

[0023] Perform pixel-level gradient analysis on the combined complete bounding box and use image gradient detection to identify discontinuous areas of the object boundary;

[0024] Set the gradient threshold W, mark the area with gradient value greater than or equal to W as the potential non-fitting phenomenon area, and iteratively adjust the shape of the boundary through the active contour model for the identified non-fitting phenomenon area;

[0025] Calculate the fit score S of the adjusted bounding box, evaluate the overlapping area between the bounding box and the true contour of the object, set the fit judgment threshold P, if the fit score is greater than or equal to P, it means that the bounding box is optimized to high precision, then output the bounding box, if it is less than P, continue iterative optimization;

[0026] Use the dilation operation to expand the boundary, and then use the erosion operation to gradually reduce the object boundary after expanding the original boundary points, and remove the noise on the boundary to obtain the optimized bounding box.

[0027] As a preferred solution of the channel occupancy monitoring method described in the present invention, wherein: the calculation of the volume of the occupied object based on the boundary detection result refers to dividing the three-dimensional contour of the occupied object into N triangular facets, calculating the area of ​​each triangle through the boundary points of the object, and calculating the total volume of the occupied object based on the area and height of each triangular facet, real-time monitoring of the changes of occupied objects in the fire channel and adjusting the volume information of the occupied objects.

[0028] As a preferred solution of the channel occupancy monitoring method of the present invention, wherein: taking different governance measures according to the volume calculation results refers to using the SVM model as a classification model, dividing the occupied objects into three violation levels as the target labels of the model;

[0029] The target labels include minor violations, moderate violations, and severe violations;

[0030] The pre-processed historical object volume, position and shape data are divided into a training set and a test set. The labeled training data set is used to train the SVM model. The SVM model parameters are optimized through cross-validation and grid search. After each training cycle, the model performance is evaluated using the test set. If there is no significant change in the verification difference after the training cycle, the model training is stopped to obtain the trained SVM model. The volume, position and shape data of the real-time object are input into the SVM model for classification to obtain the violation level of the occupied object.

[0031] If the occupied object is a minor violation, the first-level governance measure will be triggered, and a voice reminder will be issued through the speaker installed in the passage to notify the on-site personnel to remove the occupied object in time, and the relevant management personnel will be notified through SMS and applications to handle it;

[0032] If the occupied object is a moderate violation, the secondary governance measures will be triggered, automatically marking and recording the image and video of the object, and sending the occupancy situation to the management center, notifying security personnel and property personnel to go to the site and manually remove the occupied object;

[0033] If the occupied object is a serious violation, the third-level governance measures will be triggered, and a high-level alarm will be issued immediately to indicate serious occupation, and the emergency management team will be notified to deal with the object immediately. If the fire passage is equipped with automatic cleaning equipment, a remote control command will be triggered to automatically clear the occupied object;

[0034] The high-level alarm includes sirens and red flashing lights;

[0035] After each level of governance measures is initiated, the status of the objects is continuously monitored and the removal status is regularly checked. The volume calculation results, governance measures and corresponding processing status of each time are stored in the database.

[0036] Another object of the present invention is to provide a channel occupancy monitoring system, which comprises:

[0037] A data acquisition module, used to collect multimodal data of the initial fire channel and generate an initial three-dimensional model after preprocessing the collected multimodal data;

[0038] An object recognition module is used to compare the changes of real-time multi-dimensional data and initial three-dimensional model data to identify occupied objects in the fire passage and generate corresponding preliminary bounding boxes;

[0039] A boundary optimization module is used to optimize the bounding box of the occupied object to generate an optimized bounding box and perform real-time volume calculation of the occupied object;

[0040] The classification management module is used to classify occupied objects using the SVM model and then take different management measures.

[0041] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of a channel occupancy monitoring method when executing the computer program.

[0042] A computer-readable storage medium stores a computer program, which implements the steps of a channel occupancy monitoring method when executed by a processor.

[0043] The beneficial effects of the present invention are as follows: the present invention collects multi-dimensional data in the fire passage and maps the data to a spatial coordinate system to generate an initial three-dimensional model. Based on the change of the three-dimensional model, the object bounding box is generated and optimized to capture the rotation information of the object, identify the mutation points of the boundary and correct the discontinuous areas, thereby generating a bounding box that is more in line with the real shape of the object. The optimized bounding box reflects the precise outline of the object, improves the recognition accuracy and stability of the system in complex scenarios, and effectively reduces the false alarm rate and latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0045] Figure 1 Schematic diagram of the flow of the channel occupancy monitoring method.

[0046] Figure 2 Schematic diagram of the operation flow of the channel occupancy monitoring method.

[0047] Figure 3 This is a structural diagram of the channel occupancy monitoring system. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0051] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a channel occupancy monitoring method, the channel occupancy monitoring method comprising:

[0052] S1, collecting and preprocessing the initial multi-dimensional data of the fire passage, and mapping the multi-dimensional data to a spatial coordinate system to generate an initial three-dimensional model;

[0053] Specifically, collecting the initial multi-dimensional data of the fire channel and performing preprocessing means deploying RGB cameras, lidars and depth sensors in the fire channel, collecting the color information, two-dimensional contours, three-dimensional point cloud data and depth maps of the initial objects in the fire channel, and preprocessing and time synchronization calibrating the collected multi-dimensional data.

[0054] The color information of the RGB camera can help identify objects of different materials, the point cloud data of the LiDAR provides the three-dimensional shape information of the object, and the depth sensor further supplements the height and distance information of the object. Through the fusion of multi-dimensional data, the system can generate a high-precision three-dimensional model to better identify the size, position and shape of occupied objects. The purpose of preprocessing is to clean up noisy data and standardize sensor output to ensure that the subsequent model generation and recognition process are more accurate. Specifically, data preprocessing includes removing invalid data points, filling in missing data, and aligning and standardizing data from different sensors. Through the preprocessing steps, the system can reduce errors in sensor data collection and improve data accuracy. Time synchronization calibration can ensure the spatiotemporal consistency of multi-dimensional data, improve the real-time and accuracy of system recognition, and avoid data mismatch and misrecognition due to time differences.

[0055] Further, the multi-dimensional data is mapped into a spatial coordinate system to generate an initial three-dimensional model, the depth map and the point cloud data are mapped into a unified three-dimensional coordinate system, and the mapped depth map and point cloud data are combined to form a three-dimensional geometric shape of the object;

[0056] The color and two-dimensional boundary of each object are extracted through the two-dimensional contours of the objects collected in real time. The two-dimensional contours generated by the RGB image and the depth map are combined to generate an initial three-dimensional model of the fire passage. After standardization, the three-dimensional model is used as the standard state of the fire passage without occupied objects.

[0057] By combining the depth map and point cloud data, the system can capture the three-dimensional geometric shape of the object and generate a complete initial three-dimensional model, which solves the problem that traditional two-dimensional image recognition cannot accurately reflect the spatial position and shape of the object, and enables the system to better cope with complex fire passage scenes, such as multiple objects superimposed or partially blocked. After generating the initial three-dimensional model, the system extracts the color and boundary information of each object through the two-dimensional contour data of the object collected in real time, and combines it with the depth map. In this way, the system can describe the object more comprehensively, not only accurately identify the boundary and shape of the object, but also obtain surface features such as color, which can realize accurate identification of occupied objects, thereby reducing false alarms and missed alarms. The generated three-dimensional model is standardized to obtain the standard state of no occupied objects in the fire passage, which can be quickly compared and identified when collecting data in real time, and can not only detect the occupancy of larger objects, but also identify smaller objects or subtle occupancy changes. This improvement in accuracy ensures that even small or short-lived occupied objects in the fire passage will not be ignored, thereby greatly improving safety and reliability.

[0058] S2, comparing the real-time multi-dimensional data with the data in the initial three-dimensional model to identify the changed occupied objects and generate a preliminary bounding box, and optimizing the detected occupied object boundaries;

[0059] Specifically, comparing the real-time multi-dimensional data with the data in the initial three-dimensional model to identify the changed occupied objects and generate a preliminary bounding box refers to collecting the real-time multi-dimensional data in the fire passage and pre-processing it, then comparing the real-time collected data with the initial three-dimensional model frame by frame, calculating the difference between the position, color and depth of each point and the corresponding point in the initial three-dimensional model through the ICP algorithm, comparing all points one by one, and obtaining a comprehensive change index after weighted fusion of the obtained difference values, setting the change threshold K through the Otsu's method, and marking the area with a change index greater than or equal to K as a change area;

[0060] The Kmeans++ clustering algorithm is used to cluster the change points in the marked change area. The clustering data k is set according to the change area detected in real time. All marked change points are assigned to different clustering categories through iterative optimization. For each clustering result, a preliminary bounding box of the occupied object is generated.

[0061] By comparing real-time multi-dimensional data with the initial three-dimensional model, it is possible to efficiently and accurately identify newly added or moved occupied objects in the fire passage. The ICP algorithm has extremely high accuracy in processing complex three-dimensional environments, and can effectively align real-time data with each point in the model, reducing the possibility of false positives and false negatives. This is crucial for the detection of objects occupying the passage, because there may be multiple occupied objects in the passage, and the position of the objects is not fixed. Through Kmeans++ clustering, the system can classify the detected change points and generate more accurate occupied object bounding boxes based on these clustering results. Compared with the simple threshold division method, the clustering algorithm can effectively capture the shape and boundary features of the object, so that the generated bounding box is more consistent with the actual object shape. The iterative optimization characteristics of the clustering algorithm enable it to adapt to the dynamic changes of objects. In the case of the movement or shape change of the occupied object, the clustering algorithm can dynamically adjust the bounding box, making the system's detection results more accurate. By setting a reasonable threshold, the system can focus on the obvious object occupancy change area, avoid false alarms due to subtle noise or non-critical changes, and significantly improve the overall detection accuracy of the system.

[0062] Further, optimizing the detected occupied object boundaries means detecting the objects in the preliminary bounding box and calculating the rotation angle of each vertex of the object boundary. Identify the rotation information of an object:

[0063] Existing techniques calculate angles by simple coordinate differences:

[0064]

[0065] This formula is applicable to stationary objects or linear scenes, but it cannot handle the influence of dynamic factors such as object speed, acceleration, inertia, etc. In order to consider the influence of object movement on the angle, we introduced speed and acceleration correction terms into the original formula and used exponential decay function to control these dynamic parameters:

[0066]

[0067] The exponential decay function can smoothly process the angle changes under high speed and high acceleration conditions, ensuring that the angle calculation is accurate even in dynamic scenes. In order to adapt to complex rotation scenes, the correction term of rotational inertia is added, and the influence of the inertia of the object on the angle calculation is considered to obtain the final rotation angle calculation formula:

[0068]

[0069] In the formula, is the distance between two boundary points in the y-axis direction, is the distance between two boundary points in the x-axis direction, and is the velocity of the object in the x and y directions, calculated from the rate of change of the object's coordinates over time, usually using the finite difference method, and is the acceleration of the object in the x and y directions, calculated from the rate of change of the object's velocity with time, using finite differences or second-order derivatives, is the exponential decay coefficient, which controls the degree of decay of velocity and acceleration to angle changes. It can be determined by experiments or simulations. I(a) is the rotational inertia of the object, which describes the resistance of the object to rotational changes. It is determined by the mass and shape of the object. a is the angular velocity of the object. r is the rotation radius, which represents the distance of the object relative to the rotation axis. It is calculated based on the geometric information of the object. () is the inverse tangent function;

[0070] By correcting the rotational inertia, the system can accurately consider the influence of the mass distribution of the object and the rotation radius on the angle, which is particularly important for calculating the rotation angle of objects with large mass or complex geometry.

[0071] Mark the angular position of the rotating object based on the calculated angle information of the object boundary, and calculate the angle change of each boundary vertex Perform calculations to identify angle mutation points:

[0072] In the prior art, the sudden change of the boundary is detected by simply using the difference between adjacent angles:

[0073] ,

[0074] Although this formula can simply calculate angle mutations, it is easily affected by noise and is not sensitive enough to angle changes at complex boundaries. In order to improve the accuracy of mutation detection at the boundaries of complex objects, a convolution operation is introduced to smooth the angle changes using a time window: , convolution can effectively reduce noise, especially in the case of drastic angle changes, it can smooth the angle changes and make mutation detection more robust. In order to reflect the dynamic changes of angles over time, the time derivative of the angle is added to the original formula to obtain the final angle mutation detection formula:

[0075]

[0076] Where K(t-τ) is the Gaussian convolution kernel, which is used to smooth the time variation of the angle. It is a dimensionless function and represents the smoothing window of the angle variation. The reasonable convolution kernel parameters can be measured experimentally. is the angle of the i-th boundary point at time t. By capturing the boundary position changes of the object at different time points, the boundary angle is calculated. is the angle of the i+1th boundary point at time t, calculated by the coordinate difference of two adjacent points, is the time derivative of the angle of the i-th boundary point, which indicates the rate of change of the angle over time and reflects the dynamic characteristics of the angle change. It is usually obtained by numerically differentiating the time series. and It is the boundary of the time interval, which is used to calculate the time window of the convolution integral, reflecting the angle change of the object within the time period. It is set according to the actual application scenario. dτ is the small change of the integral variable τ, which is used to represent the change of the time variable in the convolution operation;

[0077] The time derivative reflects the changing trend of the angle over time, making the system more sensitive to the dynamic changes of the angle when dealing with complex rotation scenes;

[0078] Analyze the angle change of each mutation point. When the angle mutations of multiple adjacent vertices occur in the same area, there is discontinuity at the boundary of the area. Record each confirmed mutation point and discontinuous area.

[0079] The complex exponential encoding formula is used to encode the rotation angle of each mutation point, and the angle information of the object boundary is converted into a smooth expression in the complex domain:

[0080]

[0081] Where z is the encoded complex value, θ is the angle of rotation of the object, w is the angular frequency, which is used to encode the frequency of the object's rotation angle and control the smoothness of the angle. It is determined through experiments or debugging according to the specific scenario, and j is the imaginary unit;

[0082] According to the boundary characteristics of the object (the shape complexity and rotation degree of the boundary, if the rotation angle changes greatly, set a high angular frequency to accelerate angle smoothing. For objects with complex shapes, set a low angular frequency to retain boundary details and avoid over-speed smoothing) adjust the angular frequency in real time to control the smoothness of the encoding;

[0083] Recalculate and decode the rotation angle information using the inverse decoding formula of the complex exponential encoding:

[0084]

[0085] Where ln() is the logarithm of the complex number, which is used to extract the phase information in the complex number, z is the complex number, j is the imaginary unit, and w is the angular frequency;

[0086] Reconstruct the boundary segments based on the rotation angle of each vertex and the coordinates of the adjacent vertices:

[0087]

[0088] In the formula, and are the coordinates of the adjacent vertices, and L is the length of the boundary segment;

[0089] After smoothness detection of the newly generated boundary segments, all boundary segments are reassembled to generate a complete object bounding box;

[0090] Perform pixel-level gradient analysis on the combined complete bounding box and use image gradient detection to identify discontinuous areas of the object boundary:

[0091]

[0092] In the formula, It's a pixel The gradient value at , I is the image brightness value, and Represent the image gradients in the horizontal and vertical directions respectively;

[0093] The gradient threshold W is set based on the statistical distribution of the gradient of the image, and the area with a gradient value greater than or equal to W is marked as a potential non-fitting phenomenon area. For the identified non-fitting phenomenon area, the shape of the boundary is iteratively adjusted through the active contour model;

[0094] Calculate the fit score S of the adjusted bounding box, evaluating the overlap area between the bounding box and the true contour of the object:

[0095]

[0096] Where A is the true boundary of the object, B is the adjusted bounding box, represents the overlapping area between the two. represents the area of ​​the union of the two;

[0097] The fitting judgment threshold P is set according to the needs of the specific application scenario. If the fitting score is greater than or equal to P, it means that the bounding box is optimized to high precision, and the bounding box is output. If it is less than P, iterative optimization continues;

[0098] In fire passage occupancy monitoring, due to safety reasons, any possible occupancy behavior needs to be responded to quickly, so the P value can be set lower to ensure that the system can quickly generate a bounding box and trigger an early warning when any possible occupancy object appears. In some applications with higher fault tolerance, the P value can be set higher to reduce false alarms.

[0099] Use the dilation operation to expand the boundary, and then use the erosion operation to gradually reduce the object boundary after expanding the original boundary points, and remove the noise on the boundary to obtain the optimized bounding box.

[0100] Based on the initially generated bounding box, by calculating the rotation angle of each vertex of the object boundary, the rotation state of the object can be accurately identified, and the rotation change of the object can be captured, especially when the object moves or rotates dynamically, the traditional static bounding box cannot accurately describe the actual shape of the object. By calculating the rotation angle, the system can generate a bounding box that fits the object better, avoiding the boundary dislocation or distortion caused by the movement of the object in the static method. This technology is particularly suitable for monitoring the occupancy of dynamic objects to ensure the accuracy of boundary identification. The angle change of the boundary point is smoothed using the Gaussian convolution kernel, and the mutation points in the boundary of the object can be detected. When the angle changes of multiple adjacent vertices mutate at the same time, the system can identify that there is discontinuity in the boundary of the area. By marking the mutation points and discontinuous areas, the system can further optimize the complex boundaries of the object to generate smoother and more accurate boundary segments. The rotation angle information of the object is converted through complex exponential coding, and the boundary information can be smoothly expressed in the complex domain. Through coding, the system can eliminate small errors between boundary points to generate smoother boundary lines. The encoded data can be decoded in reverse to restore the rotation angle information while maintaining the continuity of the boundary. Complex exponential coding eliminates small disturbances during the rotation of the object boundary and achieves smoother bounding box generation. This coding technology is particularly suitable for processing objects with complex boundaries and irregular contours, significantly improving the recognition accuracy of the boundary.

[0101] Through pixel-level gradient analysis, the system can accurately identify the non-fitting phenomenon areas of the object boundary, that is, the areas where the boundary is discontinuous. The active contour model (also known as the Snakes model) is used to iteratively adjust the boundaries of these areas so that the boundaries can fit the real contour of the object more closely. The active contour model shows strong adaptability when processing objects with complex shapes, and can automatically adjust the boundary position to obtain a higher-precision bounding box. Compared with the traditional boundary fitting method, the active contour model can dynamically adjust the boundary according to the actual shape of the object, greatly improving the fit between the bounding box and the real contour of the object. In the final stage of boundary optimization, the system combines expansion and erosion operations to further refine the object boundary. The expansion operation allows the boundary to be expanded to cover the subtle structure or complex morphology on the surface of the object, while the erosion operation is used to remove noise points on the boundary to ensure the accuracy of the boundary segment. This process can not only retain the real morphological characteristics of the object, but also eliminate boundary anomalies caused by sensor noise or errors. The combination of expansion and erosion operations provides a refined processing method for the final generation of the bounding box, ensuring that the bounding box can accurately reflect the contour of the object while eliminating noise and unnecessary errors.

[0102] S3, calculate the volume of the occupied object according to the boundary detection result, and take different governance measures according to the volume calculation result;

[0103] Specifically, calculating the volume of the occupied object according to the boundary detection result means dividing the three-dimensional contour of the occupied object into N triangular patches through the Delaunay triangulation algorithm, and calculating the area of ​​each triangle through the boundary points of the object:

[0104]

[0105] Where F is the area of ​​each triangle patch, v is the side length of the triangle, and b is the height of each triangle patch. The side length and height of the triangle are calculated by the distance between the vertices of the triangle.

[0106] Calculate the total volume of the object based on the area and height of each triangle patch:

[0107]

[0108] Where V is the volume of the object, F is the area of ​​each triangle patch, b is the height of each triangle patch, n is the total number of triangle patches, and i is the index variable;

[0109] Monitor the changes of occupied objects in the fire passage in real time and adjust the volume information of the occupied objects.

[0110] The Delaunay triangulation algorithm has significant geometric optimization advantages. It can maximize the minimum angle of the triangle and ensure the uniformity and stability of the generated triangular mesh. This algorithm avoids the generation of long and thin triangles, ensures the smoothness and accuracy of the object boundary, and is particularly suitable for three-dimensional modeling of complex shapes and irregular objects. In the fire channel occupancy monitoring scene, the object may have an irregular shape. Through this algorithm, the three-dimensional contour of the object can be efficiently divided into multiple triangular facets with stable structures, ensuring the accuracy of subsequent volume calculations. By extracting the boundary points of the object, the vertices of the triangular facets can be accurately defined. The area of ​​the triangle is calculated by the coordinates of these vertices, which can reflect the local surface area of ​​the object. Combining the area of ​​each triangular facet with its height can effectively calculate the volume of the facet in three-dimensional space, and can accurately calculate the volume of irregular-shaped objects, avoiding the errors caused by simple geometric estimation methods. Through real-time data collection and dynamic volume updates, the system can continuously monitor the changes in occupied objects in the fire channel. Whether it is the movement of the object's position, the change of volume, or the addition of new objects, the system can respond in a timely manner by quickly calculating and updating the volume information.

[0111] Furthermore, taking different governance measures according to the volume calculation results refers to using the SVM model as a classification model and classifying the occupied objects into three violation levels as the target labels of the model;

[0112] The target labels include minor violations, moderate violations, and severe violations;

[0113] The pre-processed historical object volume, position and shape data are divided into a training set and a test set. The labeled training data set is used to train the SVM model. The SVM model parameters are optimized through cross-validation and grid search. After each training cycle, the model performance is evaluated using the test set. If there is no significant change in the verification difference after the training cycle, the model training is stopped to obtain the trained SVM model. The volume, position and shape data of the real-time object are input into the SVM model for classification to obtain the violation level of the occupied object.

[0114] If the occupied object is a minor violation, the first-level governance measure will be triggered, and a voice reminder will be issued through the speaker installed in the passage to notify the on-site personnel to remove the occupied object in time, and the relevant management personnel will be notified through SMS and applications to handle it;

[0115] If the occupied object is a moderate violation, the secondary governance measures will be triggered, automatically marking and recording the image and video of the object, and sending the occupancy situation to the management center, notifying security personnel and property personnel to go to the site and manually remove the occupied object;

[0116] If the occupied object is a serious violation, the third-level governance measures will be triggered, and a high-level alarm will be issued immediately to indicate serious occupation, and the emergency management team will be notified to deal with the object immediately. If the fire passage is equipped with automatic cleaning equipment, a remote control command will be triggered to automatically clear the occupied object;

[0117] The high-level alarm includes sirens and red flashing lights;

[0118] After each level of governance measures is initiated, the status of the objects is continuously monitored and the removal status is regularly checked. The volume calculation results, governance measures and corresponding processing status of each time are stored in the database.

[0119] By combining multidimensional data such as the volume, position and shape of occupied objects, SVM can automatically classify objects into mild, moderate and severe violations, which is more intelligent and efficient than traditional manual classification or simple rule classification. The introduction of the SVM model can not only handle complex multidimensional features, but also effectively avoid the occurrence of misclassification, especially when facing irregular objects, the classification accuracy of the SVM model is particularly outstanding. By hierarchical processing of occupied objects and adopting flexible governance measures, the response capability of the channel monitoring system is greatly improved. The first-level governance measures effectively avoid excessive response and save management costs and time. The second-level governance measures ensure the complete record of information, which is convenient for post-review and management follow-up. The third-level governance measures can greatly improve the intelligent level of channel occupancy management, reduce the burden of manual intervention, and provide managers with efficient and accurate response means. After each level of governance measures is activated, the system will continuously monitor the status of the object to ensure that the object is removed. This continuous monitoring function can prevent the recurrence of violations. By recording the governance process and results of each time, the system can automatically learn and optimize, and gradually improve the efficiency of responding to different types of violations. This not only ensures the safety and smooth flow of fire escape routes, but also provides management departments with comprehensive historical data support, facilitating the formulation of future safety management and emergency response strategies.

[0120] Example 2, reference Figure 3 , which is a second embodiment of the present invention, and which is different from the previous embodiment, provides a channel occupancy monitoring system, which includes:

[0121] A data acquisition module, used to collect multimodal data of the initial fire channel and generate an initial three-dimensional model after preprocessing the collected multimodal data;

[0122] An object recognition module is used to compare the changes of real-time multi-dimensional data and initial three-dimensional model data to identify occupied objects in the fire passage and generate corresponding preliminary bounding boxes;

[0123] A boundary optimization module is used to optimize the bounding box of the occupied object to generate an optimized bounding box and perform real-time volume calculation of the occupied object;

[0124] The classification management module is used to classify occupied objects using the SVM model and then take different management measures.

[0125] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0126] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0127] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0128] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit with a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit with a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

Claims

1. A channel occupancy monitoring method, characterized in that: include, Collect and pre-process the initial multi-dimensional data of the fire escape passage, and map the multi-dimensional data into a spatial coordinate system to generate an initial three-dimensional model; Compare the real-time multi-dimensional data with the data in the initial three-dimensional model to identify the changed occupied objects and generate preliminary bounding boxes, and optimize the detected occupied object boundaries; The volume of the occupied object is calculated based on the boundary detection results, and different governance measures are taken based on the volume calculation results.

2. The channel occupancy monitoring method according to claim 1, characterized in that: The collecting of initial multi-dimensional data of the fire passage and preprocessing thereof refers to deploying RGB cameras, lidars and depth sensors in the fire passage, collecting color information, two-dimensional contours, three-dimensional point cloud data and depth maps of objects in the initial fire passage, and preprocessing and time synchronization calibration of the collected multi-dimensional data.

3. The channel occupancy monitoring method according to claim 2, characterized in that: The mapping of the multi-dimensional data into a spatial coordinate system to generate an initial three-dimensional model maps the depth map and the point cloud data into a unified three-dimensional coordinate system, and combines the mapped depth map and the point cloud data to form a three-dimensional geometric shape of the object; The color and two-dimensional boundary of each object are extracted through the two-dimensional contours of the objects collected in real time. The two-dimensional contours generated by the RGB image and the depth map are combined to generate an initial three-dimensional model of the fire passage. After standardization, the three-dimensional model is used as the standard state of the fire passage without occupied objects.

4. The channel occupancy monitoring method according to claim 3, characterized in that: The comparing of the real-time multi-dimensional data and the data in the initial three-dimensional model to identify the changed occupied objects and generate a preliminary bounding box refers to collecting the real-time multi-dimensional data in the fire passage and performing pre-processing before comparing the real-time collected data with the initial three-dimensional model frame by frame, calculating the difference between the position, color and depth of each point and the corresponding point in the initial three-dimensional model through the ICP algorithm, comparing all points one by one, weighted fusion of the obtained difference values ​​to obtain a comprehensive change index, setting a change judgment threshold K, and marking the area with a change index greater than or equal to K as a change area; The Kmeans++ clustering algorithm is used to cluster the change points in the marked change area. The clustering data k is set according to the change area detected in real time. All marked change points are assigned to different clustering categories through iterative optimization. For each clustering result, a preliminary bounding box of the occupied object is generated.

5. The channel occupancy monitoring method according to claim 4, characterized in that: The optimization of the detected occupied object boundary refers to detecting the object in the preliminary bounding box and calculating the rotation angle of each vertex of the object boundary. Identify the rotation information of an object: In the formula, is the distance between two boundary points in the y-axis direction, is the distance between two boundary points in the x-axis direction, and is the velocity of the object in the x and y directions, and is the acceleration of the object in the x and y directions, is the exponential decay coefficient, I(a) is the rotational inertia of the object, a is the angular velocity of the object, r is the rotation radius representing the distance of the object relative to the rotation axis, () is the inverse tangent function; Mark the angular position of the rotating object based on the calculated angle information of the object boundary, and calculate the angle change of each boundary vertex Perform calculations to identify angle mutation points: Where K(t-τ) is the Gaussian convolution kernel, is the angle of the i-th boundary point at time t, is the angle of the i+1th boundary point at time t, is the time derivative of the angle of the ith boundary point, and is the boundary of the time interval, dτ is the small change in the integral variable τ; Analyze the angle change of each mutation point. When the angle mutations of multiple adjacent vertices occur in the same area, there is discontinuity at the boundary of the area. Record each confirmed mutation point and discontinuous area. The rotation angle of each mutation point is encoded using the complex exponential encoding formula, and the angle information of the object boundary is converted into a smooth expression in the complex domain; The smoothness of the coding is controlled by adjusting the angular frequency in real time according to the boundary characteristics of the object; Recalculate and decode the rotation angle information through the reverse decoding formula of the complex exponential encoding; Reconstruct the boundary segments according to the rotation angle of each vertex and the coordinates of the adjacent vertices; After smoothness detection of the newly generated boundary segments, all boundary segments are reassembled to generate a complete object bounding box; Perform pixel-level gradient analysis on the combined complete bounding box and use image gradient detection to identify discontinuous areas of the object boundary; Set the gradient threshold W, mark the area with gradient value greater than or equal to W as the potential non-fitting phenomenon area, and iteratively adjust the shape of the boundary through the active contour model for the identified non-fitting phenomenon area; Calculate the fit score S of the adjusted bounding box, evaluate the overlapping area between the bounding box and the true contour of the object, set the fit judgment threshold P, if the fit score is greater than or equal to P, it means that the bounding box is optimized to high precision, then output the bounding box, if it is less than P, continue iterative optimization; Use the dilation operation to expand the boundary, and then use the erosion operation to gradually reduce the object boundary after expanding the original boundary points, and remove the noise on the boundary to obtain the optimized bounding box.

6. The channel occupancy monitoring method according to claim 5, characterized in that: Calculating the volume of the occupied object based on the boundary detection results refers to dividing the three-dimensional outline of the occupied object into N triangular facets, calculating the area of ​​each triangle through the boundary points of the object, calculating the total volume of the occupied object based on the area and height of each triangular facet, monitoring the changes of occupied objects in the fire passage in real time and adjusting the volume information of the occupied objects.

7. The channel occupancy monitoring method according to claim 6, characterized in that: The taking of different governance measures according to the volume calculation results refers to using the SVM model as a classification model to classify the occupied objects into three violation levels as the target labels of the model; The target labels include minor violations, moderate violations, and severe violations; The pre-processed historical object volume, position and shape data are divided into a training set and a test set. The labeled training data set is used to train the SVM model. The SVM model parameters are optimized through cross-validation and grid search. After each training cycle, the model performance is evaluated using the test set. If there is no significant change in the verification difference after the training cycle, the model training is stopped to obtain the trained SVM model. The volume, position and shape data of the real-time object are input into the SVM model for classification to obtain the violation level of the occupied object. If the occupied object is a minor violation, the first-level governance measure will be triggered, and a voice reminder will be issued through the speaker installed in the passage to notify the on-site personnel to remove the occupied object in time, and the relevant management personnel will be notified through SMS and applications to handle it; If the occupied object is a moderate violation, the secondary governance measures will be triggered, automatically marking and recording the image and video of the object, and sending the occupancy situation to the management center, notifying security personnel and property personnel to go to the site and manually remove the occupied object; If the occupied object is a serious violation, the third-level governance measures will be triggered, and a high-level alarm will be issued immediately to indicate serious occupation, and the emergency management team will be notified to deal with the object immediately. If the fire passage is equipped with automatic cleaning equipment, a remote control command will be triggered to automatically clear the occupied object; The high-level alarm includes sirens and red flashing lights; After each level of governance measures is initiated, the status of the objects is continuously monitored and the removal status is regularly checked. The volume calculation results, governance measures and corresponding processing status of each time are stored in the database.

8. A channel occupancy monitoring system based on the channel occupancy monitoring method according to any one of claims 1 to 7, characterized in that: include, A data acquisition module, used to collect multimodal data of the initial fire channel and generate an initial three-dimensional model after preprocessing the collected multimodal data; An object recognition module is used to compare the changes of real-time multi-dimensional data and initial three-dimensional model data to identify occupied objects in the fire passage and generate corresponding preliminary bounding boxes; A boundary optimization module is used to optimize the bounding box of the occupied object to generate an optimized bounding box and perform real-time volume calculation of the occupied object; The classification management module is used to classify occupied objects using the SVM model and then take different management measures.

9. A computer device comprising: Memory and processor; The memory stores a computer program, wherein the processor implements the steps of the channel occupancy monitoring method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the channel occupancy monitoring method according to any one of claims 1 to 7 are implemented.

Images