Coal mine area personnel limit monitoring method, device, equipment, medium and product

Abstract

The invention discloses a coal mine area personnel limit monitoring method, device and equipment, a medium and a product. The method comprises the following steps: acquiring image data of a coal mine area access channel at the current moment in real time, and preprocessing to obtain processed image data; performing background detection and updating processing on the processed image data according to a background model and an image segmentation threshold value at the previous moment to obtain a foreground model and a background model; performing personnel fitting modeling on the foreground model to obtain a spatial position set of all personnel in the foreground model; and according to the spatial position set, a time sequence bipartite graph matching method is adopted to generate a personnel movement track of the coal mine area, and the real-time number of people in the coal mine area is determined so as to carry out area personnel limiting monitoring. The method can accurately position the positions of people in the foreground, avoids repeated counting or missed counting, guarantees the accurate real-time counting of people in the region, and provides reliable data support for personnel limit monitoring.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, device, equipment, medium, and product for monitoring personnel restrictions in coal mine areas. Background Technology

[0002] Coal mine area staffing limits are a comprehensive measure that determines the maximum number of workers that can be accommodated in different areas, taking into account factors such as the size of the underground space, ventilation conditions, safety equipment, and evacuation route planning. By limiting the maximum number of people in each area, the responsibilities of workers in different areas can be clearly defined, effectively preventing chaos caused by excessive gathering of people in a single area and ensuring the orderly conduct of underground production operations. Furthermore, in the event of a sudden hazard, reasonable area staffing limits can ensure the smooth implementation of evacuation route planning, reduce congestion, improve the efficiency of passage and evacuation in each area, and reduce the risk of casualties caused by disasters. However, violations of area staffing limit regulations frequently occur in coal mine production, thus requiring monitoring of staffing limits in coal mine areas. Currently, the main technologies for implementing area staffing limits in underground coal mines are carrier-free communication technology and AI image processing technology, but both have significant drawbacks and cannot meet the needs of efficient and stable staffing limit monitoring: for example, carrier-free communication technology requires the installation of base stations covering the entire production area and tags carried by personnel, resulting in a huge workload for installation and maintenance. AI image processing technology has high requirements for lighting conditions. When there is insufficient light source (such as lighting equipment failure) or backlight (such as personnel walking facing the light source) in the mine, the image quality will be severely degraded, causing the technology to fail completely and creating a blind spot for supervision. Summary of the Invention

[0003] This invention provides a method, device, equipment, medium, and product for monitoring personnel limits in coal mine areas. It utilizes computer vision and deep learning technologies to perform personnel fitting and modeling on image data of coal mine roadways to generate personnel movement trajectories in the coal mine area, thereby determining the real-time number of people in the coal mine area and completing the area-limited personnel monitoring.

[0004] To achieve the above objectives, embodiments of the present invention provide a method for monitoring personnel restrictions in coal mine areas, comprising: Real-time image data of the coal mine area's access passages is collected at the current moment, and the image data is preprocessed to obtain processed image data; Based on the background model and image segmentation threshold of the previous moment, background detection and update processing are performed on the processed image data to obtain the foreground model and background model of the processed image data. Person fitting modeling is performed on the foreground model to obtain the spatial location set of all persons in the foreground model; Based on the spatial location set, a temporal bipartite graph matching method is used to generate the personnel movement trajectory of the coal mine area. The real-time number of people in the coal mine area is determined based on the personnel movement trajectory for regional personnel limit monitoring.

[0005] As an improvement to the above scheme, if the image data includes color images and point cloud images of the coal mine area's access passage at the current moment, The step of performing background detection and update processing on the processed image data based on the background model and image segmentation threshold of the previous time step to obtain the foreground model and background model of the processed image data includes: Obtain the background model of the processed image data from the previous time step, and denote it as the baseline background model; Based on the baseline background model, grayscale image segmentation threshold and point cloud image segmentation threshold, background detection is performed on the processed image data according to the preset image segmentation formula to obtain the foreground model of the processed image data. Based on the baseline background model, grayscale image segmentation threshold, and point cloud image segmentation threshold, the background of the processed image data is updated according to a preset image update formula to obtain the background model of the processed image data.

[0006] As an improvement to the above scheme, the step of performing personnel fitting modeling on the foreground model to obtain the spatial location set of all personnel in the foreground model includes: In the foreground model, a number of point clouds are randomly selected for head fitting iterations until the number of iterations reaches the head fitting upper limit, thus obtaining the optimal head model. Several point clouds are randomly selected from the foreground model for neck fitting iteration until the number of iterations reaches the upper limit of the neck fitting, thus obtaining the optimal neck model. The optimal neck model and the optimal head model are jointly judged. If the number of iterations of the joint judgment has not reached the upper limit, the head and neck fitting iterations are repeated until the number of iterations of the joint judgment reaches the upper limit, and the optimal head and neck joint model is obtained. Delete the point cloud corresponding to the best head and neck joint model in the foreground model, and traverse the point cloud in the foreground model until all the best head and neck joint models in the foreground model are obtained. Based on all optimal head and neck joint models, determine the set of spatial locations of all persons in the foreground model.

[0007] As an improvement to the above scheme, the step of randomly selecting several point clouds in the foreground model for head fitting iterations until the number of iterations reaches the upper limit of the head fitting, to obtain the optimal head model, includes: Several point clouds are randomly selected in the foreground model for head fitting iteration to obtain a head quadratic form that meets the preset head conditions. Calculate the number of points in the head quadratic form. If there is no optimal head quadratic form, or if there is an optimal head quadratic form and the number of points in the currently fitted head quadratic form is higher than the number of points in the optimal head quadratic form, then the currently fitted head quadratic form is recorded as the optimal head quadratic form. If the number of iterations does not reach the head fitting upper limit, the head fitting is repeated until the number of iterations reaches the head fitting upper limit to obtain the optimal head model.

[0008] As an improvement to the above scheme, the step of randomly selecting several point clouds in the foreground model for neck fitting iterations until the number of iterations reaches the upper limit of the neck fitting, to obtain the optimal neck model, includes: Several point clouds are randomly selected in the foreground model for neck fitting iteration to obtain a neck quadratic form that meets the preset neck conditions. Calculate the number of point clouds of the neck quadratic form. If there is no optimal neck quadratic form, or there is an optimal head quadratic form and the number of point clouds of the currently fitted neck quadratic form is higher than the number of point clouds of the optimal neck quadratic form, then the currently fitted neck quadratic form is recorded as the optimal neck quadratic form. If the number of iterations does not reach the upper limit of the neck fitting, the neck fitting is repeated until the number of iterations reaches the upper limit of the neck fitting, thus obtaining the optimal neck model.

[0009] As an improvement to the above scheme, the step of generating personnel movement trajectories in the coal mine area using a temporal bipartite graph matching method based on the spatial location set, and determining the real-time number of people in the coal mine area based on the personnel movement trajectories for regional personnel limit monitoring, includes: Obtain the set of spatial positions of all personnel in the foreground model of the access passage to the coal mine area at the previous moment, and denote it as the first spatial position; Based on the spatial location and the first spatial location, calculate the Euclidean distance between the centers of the spatial locations of each person in the foreground model at two adjacent moments; Based on the Euclidean distance, a time-series bipartite graph matching method is used to generate the personnel movement trajectory in the coal mine area; The real-time number of people in the coal mine area is determined based on the movement trajectory of the personnel. If the real-time number of people exceeds the maximum allowed number of people in the coal mine area, an alarm is triggered to complete the area personnel restriction monitoring.

[0010] To achieve the above objectives, embodiments of the present invention provide a coal mine area personnel restriction monitoring device, comprising: The image data acquisition module is used to acquire image data of the coal mine area access passage at the current moment in real time, and to preprocess the image data to obtain processed image data; The image data processing module is used to perform background detection and update processing on the processed image data based on the background model and image segmentation threshold of the previous time step, so as to obtain the foreground model and background model of the processed image data. The spatial location determination module is used to perform personnel fitting modeling on the foreground model to obtain the spatial location set of all personnel in the foreground model. The real-time number of people determination module is used to generate the movement trajectory of people in the coal mine area by using the temporal bipartite graph matching method based on the spatial location set, and to determine the real-time number of people in the coal mine area based on the movement trajectory of people, so as to carry out regional personnel limit monitoring.

[0011] To achieve the above objectives, this invention provides a coal mine area personnel restriction monitoring device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the above-described coal mine area personnel restriction monitoring method.

[0012] To achieve the above objectives, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the above-described coal mine area personnel restriction monitoring method.

[0013] To achieve the above objectives, embodiments of the present invention also provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps of the above-described coal mine area personnel restriction monitoring method.

[0014] Compared with existing technologies, the present invention discloses a method, device, equipment, medium, and product for monitoring personnel limits in coal mine areas. This method involves real-time acquisition of image data from coal mine access channels, preprocessing the image data to obtain processed image data, performing background detection and update processing on the processed image data based on the background model and image segmentation threshold from the previous moment to obtain a foreground and background model, and then performing personnel fitting modeling on the foreground model to obtain a spatial location set of all personnel in the foreground model. Based on the spatial location set, a temporal bipartite graph matching method is used to generate personnel movement trajectories in the coal mine area. These trajectories are then used to determine the real-time number of people in the coal mine area for personnel limit monitoring. By utilizing computer vision and deep learning technologies to perform personnel fitting modeling on image data from coal mine roadways to generate personnel movement trajectories, the real-time number of people in the coal mine area can be determined. This method can accurately locate the positions of personnel in the foreground, and through trajectory matching, avoids double counting or undercounting, ensuring accurate real-time personnel statistics and providing reliable data support for personnel limit monitoring. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a method for monitoring personnel limits in coal mine areas, as provided in an embodiment of the present invention. Figure 2 This is a flowchart of a multimodal data background modeling process provided in an embodiment of the present invention; Figure 3 This is a flowchart of a head and neck modeling process provided in an embodiment of the present invention; Figure 4 This is a trajectory timing bipartite diagram provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a coal mine area personnel restriction monitoring device provided in an embodiment of the present invention; Figure 6 This is a structural block diagram of a coal mine area personnel restriction monitoring device provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] It should be noted that the terms "comprising" and "specific" in this invention, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0018] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for monitoring personnel limits in coal mine areas according to an embodiment of the present invention. The method includes: S1, Real-time acquisition of image data of the coal mine area access passage at the current moment, and preprocessing of the image data to obtain processed image data; S2, perform background detection and update processing on the processed image data based on the background model and image segmentation threshold of the previous time step to obtain the foreground model and background model of the processed image data; S3, perform personnel fitting modeling on the foreground model to obtain the spatial location set of all personnel in the foreground model; S4. Based on the spatial location set, the temporal bipartite graph matching method is used to generate the personnel movement trajectory of the coal mine area. Based on the personnel movement trajectory, the real-time number of people in the coal mine area is determined for regional personnel limit monitoring.

[0019] For example, multimodal cameras can be installed in key areas of underground coal mines, such as access roadways (e.g., mining area entrances, working face passages). Two types of equipment can be selected, ensuring that the lenses cover the entire cross-section of the roadway without blind spots. Even in complex lighting conditions such as insufficient light sources (e.g., localized lighting failures) or backlighting (e.g., direct light from miners' lamps), the cameras can reliably identify personnel and count heads, avoiding monitoring failures due to lighting issues. The two types are: a binocular structured light camera (based on binocular stereo vision, extracting target 3D spatial information through parallax calculation of left and right cameras); and a millimeter-wave radar camera (calculating the distance between the target and equipment based on the time difference between millimeter-wave emission and reflection, generating 3D spatial information). The equipment synchronously collects real-time image data (color images and point cloud images) of the coal mine access roadways, converting the color images into... Converting to grayscale using grayscale algorithms (such as weighted average) To simplify subsequent background modeling calculations; due to factors such as dust and equipment obstruction in coal mines, point cloud images contain holes (areas that cannot be imaged). The nearest neighbor method is used to interpolate the holes: for the location of a hole, the 3D coordinates of the nearest non-hole pixel are searched and assigned to the hole pixel to obtain a complete point cloud image. ; Match the pixels of the color image to the pixels of the point cloud image one-to-one (i.e., the pixels of the color image) Pixels of the point cloud (corresponding to the same spatial location), the two are joined together according to the third dimension. This generates color and spatial feature maps (multimodal feature maps). Based on the background model and image segmentation threshold from the previous time step, background detection and updating are performed on the processed image data to obtain the foreground and background models of the processed image data. For example... Figure 2 As shown, Figure 2 This is a flowchart of a multi-modal data background modeling process provided in an embodiment of the present invention; each time a color image is collected... With point cloud map Next, it first checks if initialization is complete. If not, background initialization is performed. After initialization is complete, background initialization is stopped, and background detection begins. After background detection, the detection results are output, and then background updates are performed. Finally, data is reacquired, and the process returns to the beginning. Personnel fitting modeling is performed on the foreground model to obtain the spatial location set of all personnel in the foreground model. Based on the spatial location set, a temporal bipartite graph matching method is used to generate the personnel movement trajectory of the coal mine area. The real-time number of people in the coal mine area is determined based on the personnel movement trajectory for area personnel control monitoring.

[0020] Understandably, point cloud images are acquired based on binocular stereo vision or millimeter-wave radar principles, and their three-dimensional spatial information is unaffected by lighting conditions. The multimodal data formed by stitching color images and point cloud images can compensate for the lack of color features in color images when lighting is poor through the spatial characteristics of point cloud images. Even in complex lighting scenarios such as insufficient light sources in coal mines (e.g., local lighting failures) and backlighting (e.g., direct light from a miner's lamp), it can still stably identify personnel and count the number of people, avoiding monitoring failures caused by lighting problems. It has strong anti-interference capabilities and adapts to the complex lighting environment in coal mines, improving monitoring accuracy.

[0021] By fitting and modeling the foreground model with people, interference from non-person targets (such as tools and rocks) can be eliminated, and the set of people's spatial locations can be accurately obtained. The temporal bipartite graph matching method is used to generate people's movement trajectories, which can effectively associate the same person in consecutive frames and avoid counting the same person repeatedly. At the same time, the number of people entering and leaving can be accurately counted by the trajectory direction, reducing the number of missed counts. This makes the accuracy of personnel positioning and trajectory matching high, reduces the error in the number of people count, and ensures the reliability of the personnel limit judgment.

[0022] Specifically, if the image data includes color images and point cloud images of the coal mine area's access passage at the current moment, Then step S2 includes: S21, Obtain the background model of the processed image data from the previous time step, denoted as the baseline background model; S22, based on the baseline background model, grayscale image segmentation threshold and point cloud image segmentation threshold, perform background detection on the processed image data according to the preset image segmentation formula to obtain the foreground model of the processed image data; S23, based on the baseline background model, grayscale image segmentation threshold and point cloud image segmentation threshold, the background of the processed image data is updated according to the preset image update formula to obtain the background model of the processed image data.

[0023] For example, a grayscale image segmentation threshold is set based on the characteristics of the underground coal mine environment. Point cloud segmentation threshold The preset image segmentation formula is: , In the formula, For pixels Foreground / background segmentation results, when At that time, pixels Belongs to the foreground model of processed image data, when At that time, pixels This belongs to the background model of the processed image data; pixels in a grayscale image grayscale value; Pixels in the grayscale image baseline background model The baseline grayscale value; For pixels in the processed point cloud image 3D spatial coordinate vector; Pixels in the baseline background model of the point cloud image The reference three-dimensional space coordinate vector; The L2 norm of a vector.

[0024] To adapt to minor changes in the tunnel environment (such as slight equipment movement or dust accumulation), the background model needs to be updated in real time. Specifically, when the difference between the current grayscale value and the background model exceeds a threshold, the background model is fine-tuned to ensure it accurately reflects the actual environment. The preset image update formula is: , , In the formula, Pixels in the updated grayscale image baseline background model The baseline grayscale value; For the updated point cloud map baseline background model pixels The reference three-dimensional spatial coordinate vector; each acquisition and Then, update once. and They are used in background detection.

[0025] Specifically, if this is the first time the coal mine area personnel restriction monitoring method has been implemented, the method for obtaining the baseline background model includes: Continuous data collection frame( Grayscale image at any given moment Then, average and sum them to calculate the average value as the grayscale image reference background model; continuously collect data. Frame point cloud map Then, average and sum them up to calculate the average value as the baseline background model for the point cloud map; The baseline background model for grayscale images is: , The baseline background model for the point cloud map is: , In the formula, This is the baseline background model for the grayscale image, with a resolution of [resolution value missing]. This represents the background baseline grayscale value of each pixel in the grayscale image. Image height, in pixels (pixels), representing the number of pixels in the vertical direction of the image; The width of the image represents the number of pixels in the horizontal direction. Frame number; For the first Frame grayscale image; The baseline background model for the point cloud image has a resolution of [resolution value missing]. ; For the first Frame point cloud map; The recommended value is the total number of images in 10 seconds. It can be understood that the baseline background model includes a grayscale baseline background model and a point cloud baseline background model; when the first baseline background model is obtained, the background initialization is complete.

[0026] Specifically, step S3 includes: S31, randomly select several point clouds in the foreground model for head fitting iteration until the number of iterations reaches the head fitting upper limit, and obtain the best head model; S32, randomly select several point clouds in the foreground model to perform neck fitting iterations until the number of iterations reaches the upper limit of the neck fitting, and obtain the optimal neck model; S33, perform a joint judgment on the optimal neck model and the optimal head model. If the number of iterations of the joint judgment has not reached the upper limit, repeat the head and neck fitting iteration until the number of iterations of the joint judgment reaches the upper limit, and obtain the optimal head and neck joint model. S34, delete the point cloud corresponding to the best head and neck joint model in the foreground model, and traverse the point cloud in the foreground model until all the best head and neck joint models in the foreground model are obtained. S35, Based on all optimal head and neck joint models, determine the set of spatial locations of all persons in the foreground model.

[0027] For example, such as Figure 3 As shown, Figure 3 This is a flowchart of a head and neck modeling process provided in an embodiment of the present invention, which uses the point cloud map in the foreground model. Described as The ternary array has an ellipsoid for the head and a hyperboloid for the neck, both of which are quadratic surfaces; randomly selected from the point cloud of the foreground model. Point cloud (pixels) A head model is fitted to each point cloud. The fitted head surface is then evaluated to determine if it meets preset head conditions. If so, the head surface is denoted as a quadratic head form, and the number of points (interior points) of this quadratic head form is calculated. An optimal head model is then determined. If no optimal quadratic head form exists, or if such an optimal form exists and its point cloud count is higher than that of the optimal form, the fitted head form is designated as the optimal head form, and the optimal head model is saved. If the number of iterations does not reach the head fitting limit, the head fitting process is repeated until the limit is reached, resulting in the optimal head model. Similarly, points are randomly selected from the point cloud of the foreground model. Each point cloud (pixel) will A neck fitting is performed on the point cloud. It is determined whether the fitted neck surface meets the preset neck conditions. If so, the neck surface is recorded as a quadratic neck form, and the number of point clouds (interior points) of the quadratic neck form is calculated. An optimal neck determination is performed on the quadratic neck form. If no optimal quadratic neck form exists, or if the number of point clouds in the currently fitted neck form is higher than that in the optimal neck form, the currently fitted neck form is recorded as the optimal neck form and saved. If the number of iterations does not reach the upper limit of the neck fitting, the neck fitting is repeated until the upper limit is reached to obtain the optimal neck model. An optimal head and neck joint determination is performed on the optimal head and neck models. If the optimal number of iterations for the optimal head and neck joint determination does not reach the upper limit, a point cloud from the foreground model is randomly selected. Each point cloud (pixel) will The point cloud is fitted with head and neck data until the optimal number of iterations reaches the upper limit, and the optimal head and neck (optimal head and neck joint model) is output. The above process is repeated until all the optimal heads and necks in the foreground model are output. Based on the spatial positions of the point cloud corresponding to all the optimal head and neck joint models, the spatial position set of all people in the foreground model is determined.

[0028] It is worth noting that if an optimal head (neck) does not exist, it is randomly selected from the point cloud of the foreground model. One point, Approximately 1 / 1000 of the total number of ternaries; if an optimal head (neck) exists, then randomly select from the interior points of the optimal head (neck) of the foreground model. One point, Approximately 1 / 100 of the number of interior points; Let the expression for the head quadratic form be: , Let the expression for the neck quadratic form be: , In the formula, Let be the homogeneous coordinates of the point cloud, which is a 4-dimensional column vector. ,in These represent the x, y, and y coordinates of the point cloud in 3D space, respectively. The last element, 1, is used for homogeneous coordinate calculations to uniformly handle transformations such as point position and scaling. and It is a uniformly symmetric matrix, consisting of 10 unknowns. A quadratic surface (ellipsoid) model used to describe the head. A quadratic surface (hyperboloid) model used to describe the neck; for The transpose of .

[0029] It is worth noting that, and This is a Ruili quotient problem, using random selection. Solving for each point yields the following results: and Head (neck) condition judgment: Quadratic surface (ellipsoid) model Is it an ellipsoid, and do the lengths of the three semi-axes of the ellipsoid satisfy the radius range of the head? If all conditions are met, then The head is represented by a quadratic surface (ellipsoid); a quadratic surface (hyperboloid) model. Is it a single-leaf hyperbola, and do the two real and imaginary semi-axes satisfy the radius range of the neck? If all conditions are met, then This is a model of the quadratic surface (hyperboloid of a single leaf) of the neck.

[0030] Calculate the number of interior points: ellipsoid Changes in the lengths of the three real semi-axis Within this range, the point cloud is the interior point of the head; a single-leaf hyperbola Changes in the lengths of the two real semi-axis The point cloud within this range represents the interior points of the neck. Optimal head (neck) determination: If no optimal head (neck) exists, then save the current... ( The optimal head (neck) is defined as follows: If an optimal head (neck) exists, then the optimal head (neck) and... ( The system compares the number of interior points, and the one with more interior points is the optimal head (neck). For the joint determination of the optimal head and neck: calculate the center distance between the optimal head model and the optimal neck model. If the distance is between 50mm and 400mm (consistent with the physiological distance between the human head and neck), then the current person is successfully detected, and the center coordinates of the optimal head or neck representing the person's spatial position are saved. Then, all interior points of the optimal head and neck are deleted from the point cloud, and the optimal head and neck are deleted and the iteration is restarted. If the distance is outside the range, it is determined to be an invalid model, and the iteration process is repeated. After all personnel are located, the set of spatial positions of the personnel at the current moment is output.

[0031] More specifically, step S31 includes: S311, Randomly select several point clouds in the foreground model to perform head fitting iterations to obtain a head quadratic form that meets the preset head conditions; S312, calculate the number of point clouds of the head quadratic form. If there is no optimal head quadratic form, or if there is an optimal head quadratic form and the number of point clouds of the currently fitted head quadratic form is higher than the number of point clouds of the optimal head quadratic form, then the currently fitted head quadratic form is recorded as the optimal head quadratic form. S313, If the number of iterations does not reach the head fitting upper limit, then repeat the head fitting until the number of iterations reaches the head fitting upper limit to obtain the optimal head model.

[0032] For example, if an optimal head quadratic form does not currently exist, a head can be randomly selected from the foreground point cloud. points ( (Approximately 1 / 1000 of the total number of foreground point clouds, ensuring sample representativeness); if an optimal head quadratic form exists, randomly select from the interior points (points that conform to the model characteristics) of that optimal head quadratic form. points ( Approximately 1 / 100th of the number of interior points, improving modeling efficiency); The head is iteratively fitted using point clouds (quadratic surface fitting) until a quadratic head shape that meets the preset head conditions is obtained; the head shape is approximately ellipsoidal, and the quadratic shape is used. It means that, among them The homogeneous coordinates of the point cloud are obtained by randomly selecting... Construct a system of equations from points, solve the Ruili quotient problem to obtain the matrix. Determine the ellipsoidal model (head quadratic form); verify the matrix. Does the corresponding surface meet the preset header conditions? The preset header conditions are a matrix. The corresponding surface is an ellipsoid, and the lengths of the three semi-axes of the ellipsoid are within the radius of the head. (Conforming to the range of human head dimensions); if the matrix If the preset head conditions are met, the lengths of the three semi-axes of the ellipsoid are increased or decreased by 50mm respectively, and the points in the foreground point cloud within this range are counted; these are the head interior points (the number of point clouds in the head quadratic form). If no optimal head quadratic form exists, the currently fitted head quadratic form is recorded as the optimal head quadratic form, and its interior point count is recorded. If the optimal head quadratic form exists, the number of interior points of the current head quadratic form is compared with that of the optimal head quadratic form; the model with more interior points is updated as the optimal head quadratic form. If the matrix... If the preset head conditions are not met, random point selection is repeated; if the number of iterations does not reach the head fitting upper limit, random point selection is repeated until the number of iterations reaches the head fitting upper limit to obtain the optimal head model.

[0033] More specifically, step S32 includes: S321, Randomly select several point clouds in the foreground model to perform neck fitting iterations to obtain a neck quadratic form that meets the preset neck conditions; S322, calculate the number of point clouds of the neck quadratic form. If there is no optimal neck quadratic form, or there is an optimal head quadratic form, and the number of point clouds of the currently fitted neck quadratic form is higher than the number of point clouds of the optimal neck quadratic form, then the currently fitted neck quadratic form is recorded as the optimal neck quadratic form. S323, if the number of iterations has not reached the upper limit of the neck fitting, then repeat the neck fitting until the number of iterations reaches the upper limit of the neck fitting, and obtain the optimal neck model.

[0034] For example, similar to head fitting, if an optimal neck quadratic shape does not currently exist, it is randomly selected from the foreground point cloud. points ( (Approximately 1 / 1000 of the total number of foreground point clouds, ensuring sample representativeness); if an optimal neck quadratic form exists, randomly select from the interior points (points that conform to the model characteristics) of that optimal neck quadratic form. points ( Approximately 1 / 100th of the number of interior points, improving modeling efficiency); The point cloud is used to perform neck fitting iterations (quadratic surface fitting) until a quadratic neck shape that meets the preset neck conditions is obtained; the neck shape is approximately a single-leaf hyperboloid, using a quadratic shape. It means that, among them The homogeneous coordinates of the point cloud are obtained by randomly selecting... Construct a system of equations from points, solve the Ruili quotient problem to obtain the matrix. Determine the single-leaf hyperboloid model (neck quadratic form); verify the matrix. Does the corresponding surface meet the preset neck conditions? The preset neck conditions are a matrix. The corresponding surface is a hyperboloid of one sheet, and the lengths of the two real semi-axes and one imaginary semi-axe of the hyperboloid of one sheet are within the radius of the neck. (Conforming to the range of human neck dimensions); if the matrix If the preset neck condition is met, the lengths of the two real semi-axes of the single-leaf hyperboloid are increased or decreased by 20mm respectively. Points within this range in the foreground point cloud are counted; these are the neck inliers (the number of points in the neck quadratic form). If no optimal neck quadratic form exists, the currently fitted neck quadratic form is recorded as the optimal neck quadratic form, and its number of inliers is recorded. If the optimal neck quadratic form exists, the number of inliers in the current neck quadratic form is compared with that of the optimal neck quadratic form; the model with more inliers is updated to the optimal neck quadratic form. If the matrix... If the preset neck condition is not met, the random point selection is repeated; if the number of iterations does not reach the upper limit of the neck fitting, the random point selection is repeated until the number of iterations reaches the upper limit of the neck fitting, and the best neck model is obtained.

[0035] Specifically, S4 includes: S41, Obtain the set of spatial positions of all personnel in the foreground model of the access passage to the coal mine area at the previous moment, and denot it as the first spatial position; S42, based on the spatial position and the first spatial position, calculate the Euclidean distance between the centers of the spatial positions of each person in the foreground model at two adjacent moments; S43, Based on the Euclidean distance, the temporal bipartite graph matching method is used to generate the personnel movement trajectory of the coal mine area; S44. Determine the real-time number of people in the coal mine area based on the personnel movement trajectory. If the real-time number of people exceeds the maximum allowed number of people in the coal mine area, trigger an alarm to complete the area personnel restriction monitoring.

[0036] For example, such as Figure 4 As shown, Figure 4 This is a trajectory time-series bipartite graph provided in an embodiment of the present invention; let the previous time step be... At present and the next moment It is a set of spatial locations of people in three consecutive frames (time points), with each circle representing a person's spatial location; in two adjacent frames and In the equation, there exists a weight between any two circles. The Euclidean distance between the centers of the two personnel's spatial locations ,if ,Pick Then the weight matrix is ​​formed. For the weight matrix Perform maximum matching on the bipartite graph to find the matching combination with the largest total weight. Successful matches are considered to involve the same person, and the two matched circles form a trajectory, which is then extended. If no matching circle is found, the trajectory is retained. If no match is found for 3 consecutive seconds and the trajectory cannot be extended, it is deleted after counting. If no matching circle is found, the circle is preserved and a trajectory is generated; when Once the spatial location of the frame operator is generated, the above operations can be repeated.

[0037] Among them, weight The expression is: , trajectory The ternary representation performs an inner product operation between the vector from the starting point to the ending point of the trajectory and the preset direction. If the result is positive, the count is incremented by 1 (personnel count is incremented by 1); if the result is negative, the count is decremented by 1 (personnel count is decremented by 1). During the generation of personnel trajectories, the number of people entering and leaving is counted in real time. When the number of people in an area exceeds the limit, an alarm is triggered.

[0038] This invention discloses a method for monitoring personnel limits in coal mine areas. It involves real-time acquisition of image data from the mine's entrance and exit passages, preprocessing the image data to obtain processed image data, performing background detection and updating based on the previous background model and image segmentation threshold to obtain a foreground and background model, and then performing personnel fitting modeling on the foreground model to obtain a spatial location set of all personnel within the foreground model. Based on this spatial location set, a temporal bipartite graph matching method is used to generate personnel movement trajectories within the coal mine area. These trajectories are then used to determine the real-time number of personnel in the coal mine area for personnel limit monitoring. By utilizing computer vision and deep learning technologies to perform personnel fitting modeling on image data from coal mine roadways to generate personnel movement trajectories, the method accurately locates personnel positions in the foreground and avoids double counting or undercounting through trajectory matching, ensuring accurate real-time personnel statistics and providing reliable data support for personnel limit monitoring.

[0039] See Figure 5 , Figure 5 This is a schematic diagram of the structure of a coal mine area personnel restriction monitoring device 10 provided in an embodiment of the present invention. The coal mine area personnel restriction monitoring device 10 includes: Image data acquisition module 11 is used to acquire image data of the coal mine area access passage at the current moment in real time, and to preprocess the image data to obtain processed image data; Image data processing module 12 is used to perform background detection and update processing on the processed image data based on the background model and image segmentation threshold of the previous time step, so as to obtain the foreground model and background model of the processed image data. The spatial location determination module 13 is used to perform personnel fitting modeling on the foreground model to obtain the spatial location set of all personnel in the foreground model. The real-time number of people determination module 14 is used to generate the personnel movement trajectory of the coal mine area by using the temporal bipartite graph matching method based on the spatial location set, and to determine the real-time number of people in the coal mine area based on the personnel movement trajectory, so as to carry out regional personnel limit monitoring.

[0040] The coal mine area personnel restriction monitoring device 10 provided in this embodiment of the invention can realize all the processes of the coal mine area personnel restriction monitoring method of the above embodiment. The functions and technical effects of each module in the device are the same as the functions and technical effects of the coal mine area personnel restriction monitoring method of the above embodiment, and will not be repeated here.

[0041] See Figure 6 , Figure 6 This is a schematic diagram of the structure of a coal mine area personnel restriction monitoring device 20 provided in an embodiment of the present invention. The coal mine area personnel restriction monitoring device 20 of this embodiment includes: a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the above-described coal mine area personnel restriction monitoring method embodiment. Alternatively, when the processor 21 executes the computer program, it implements the functions of each module in the above-described coal mine area personnel restriction monitoring device embodiment.

[0042] For example, the computer program may be divided into one or more modules, which are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the coal mine area personnel control monitoring device 20.

[0043] The coal mine area personnel restriction monitoring device 20 can be a desktop computer, laptop, handheld computer, or cloud server, etc. The coal mine area personnel restriction monitoring device 20 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of the coal mine area personnel restriction monitoring device 20 and does not constitute a limitation on the device. It may include more or fewer components than shown in the diagram, or combine certain components, or use different components. For example, the coal mine area personnel restriction monitoring device 20 may also include input / output devices, network access devices, buses, etc.

[0044] The processor 21 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the coal mine area personnel restriction monitoring equipment 20, connecting all parts of the equipment through various interfaces and lines.

[0045] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements various functions of the coal mine area personnel restriction monitoring equipment 20 by running or executing the computer programs and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0046] If the integrated module of the coal mine area personnel restriction monitoring equipment 20 is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted from the content as required by the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0047] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0048] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute the coal mine area personnel restriction monitoring method as described above.

[0049] Furthermore, embodiments of the present invention also provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps of the coal mine area personnel restriction monitoring method described above.

[0050] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for monitoring personnel restrictions in coal mine areas, characterized in that, include: Real-time image data of the coal mine area's access passages is collected at the current moment, and the image data is preprocessed to obtain processed image data; Based on the background model and image segmentation threshold of the previous moment, background detection and update processing are performed on the processed image data to obtain the foreground model and background model of the processed image data. Person fitting modeling is performed on the foreground model to obtain the spatial location set of all persons in the foreground model; Based on the spatial location set, a temporal bipartite graph matching method is used to generate the personnel movement trajectory of the coal mine area. The real-time number of people in the coal mine area is determined based on the personnel movement trajectory for regional personnel limit monitoring.

2. The coal mine area personnel restriction monitoring method as described in claim 1, characterized in that, If the image data includes color images and point cloud images of the coal mine area's access passage at the current moment, The step of performing background detection and update processing on the processed image data based on the background model and image segmentation threshold of the previous time step to obtain the foreground model and background model of the processed image data includes: Obtain the background model of the processed image data from the previous time step, and denote it as the baseline background model; Based on the baseline background model, grayscale image segmentation threshold and point cloud image segmentation threshold, background detection is performed on the processed image data according to the preset image segmentation formula to obtain the foreground model of the processed image data. Based on the baseline background model, grayscale image segmentation threshold, and point cloud image segmentation threshold, the background of the processed image data is updated according to a preset image update formula to obtain the background model of the processed image data.

3. The coal mine area personnel restriction monitoring method as described in claim 1, characterized in that, The process of performing personnel fitting modeling on the foreground model to obtain the spatial location set of all personnel in the foreground model includes: In the foreground model, a number of point clouds are randomly selected for head fitting iterations until the number of iterations reaches the head fitting upper limit, thus obtaining the optimal head model. Several point clouds are randomly selected from the foreground model for neck fitting iteration until the number of iterations reaches the upper limit of the neck fitting, thus obtaining the optimal neck model. The optimal neck model and the optimal head model are jointly judged. If the number of iterations of the joint judgment has not reached the upper limit, the head and neck fitting iterations are repeated until the number of iterations of the joint judgment reaches the upper limit, and the optimal head and neck joint model is obtained. Delete the point cloud corresponding to the best head and neck joint model in the foreground model, and traverse the point cloud in the foreground model until all the best head and neck joint models in the foreground model are obtained. Based on all optimal head and neck joint models, determine the set of spatial locations of all persons in the foreground model.

4. The coal mine area personnel restriction monitoring method as described in claim 3, characterized in that, The process of randomly selecting several point clouds from the foreground model for head fitting iterations until the number of iterations reaches the upper limit of the head fitting, to obtain the optimal head model, includes: Several point clouds are randomly selected in the foreground model for head fitting iteration to obtain a head quadratic form that meets the preset head conditions. Calculate the number of points in the head quadratic form. If there is no optimal head quadratic form, or if there is an optimal head quadratic form and the number of points in the currently fitted head quadratic form is higher than the number of points in the optimal head quadratic form, then the currently fitted head quadratic form is recorded as the optimal head quadratic form. If the number of iterations does not reach the head fitting upper limit, the head fitting is repeated until the number of iterations reaches the head fitting upper limit to obtain the optimal head model.

5. The coal mine area personnel restriction monitoring method as described in claim 3, characterized in that, The step of randomly selecting several point clouds from the foreground model for neck fitting iterations until the number of iterations reaches the upper limit of the neck fitting, to obtain the optimal neck model, includes: Several point clouds are randomly selected in the foreground model for neck fitting iteration to obtain a neck quadratic form that meets the preset neck conditions. Calculate the number of point clouds of the neck quadratic form. If there is no optimal neck quadratic form, or there is an optimal head quadratic form and the number of point clouds of the currently fitted neck quadratic form is higher than the number of point clouds of the optimal neck quadratic form, then the currently fitted neck quadratic form is recorded as the optimal neck quadratic form. If the number of iterations does not reach the upper limit of the neck fitting, the neck fitting is repeated until the number of iterations reaches the upper limit of the neck fitting, thus obtaining the optimal neck model.

6. The coal mine area personnel restriction monitoring method as described in claim 1, characterized in that, The process of generating personnel movement trajectories in the coal mine area using a temporal bipartite graph matching method based on the spatial location set, and determining the real-time number of people in the coal mine area based on these trajectories for regional personnel control monitoring, includes: Obtain the set of spatial positions of all personnel in the foreground model of the access passage to the coal mine area at the previous moment, and denote it as the first spatial position; Based on the spatial location and the first spatial location, calculate the Euclidean distance between the centers of the spatial locations of each person in the foreground model at two adjacent moments; Based on the Euclidean distance, a time-series bipartite graph matching method is used to generate the personnel movement trajectory in the coal mine area; The real-time number of people in the coal mine area is determined based on the movement trajectory of the personnel. If the real-time number of people exceeds the maximum allowed number of people in the coal mine area, an alarm is triggered to complete the area personnel restriction monitoring.

7. A coal mine area personnel restriction monitoring device, characterized in that, include: The image data acquisition module is used to acquire image data of the coal mine area access passage at the current moment in real time, and to preprocess the image data to obtain processed image data; The image data processing module is used to perform background detection and update processing on the processed image data based on the background model and image segmentation threshold of the previous time step, so as to obtain the foreground model and background model of the processed image data. The spatial location determination module is used to perform personnel fitting modeling on the foreground model to obtain the spatial location set of all personnel in the foreground model. The real-time number of people determination module is used to generate the movement trajectory of people in the coal mine area by using the temporal bipartite graph matching method based on the spatial location set, and to determine the real-time number of people in the coal mine area based on the movement trajectory of people, so as to carry out regional personnel limit monitoring.

8. A coal mine area personnel restriction monitoring device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the coal mine area personnel restriction monitoring method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the coal mine area personnel restriction monitoring method as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product is stored in a storage medium, and the program product is executed by at least one processor to implement the steps of the coal mine area personnel restriction monitoring method as described in any one of claims 1-6.

Images