Mine full-coverage positioning method and system based on UWB and video blind spot completion

Abstract

The invention provides a mine full-coverage positioning method and system based on UWB and video blind spot completion, and the method comprises the steps: determining the positions of a key monitoring region and a blind region, laying a monitoring camera, and adjusting the angle of the monitoring camera; staff photos and personal information are collected, a staff identification model is constructed by using YOLO, and lightweight processing is carried out; calculating position information of the personnel according to an identification result of the personnel identification model by using a distance measurement algorithm based on a monocular imaging principle; the real-time information of the position of the person is dynamically calculated by adopting an edge calculation method, and the position of the person is deduced and visualized in real time. According to the invention, full-range positioning of the mining area can be realized, and the positions of personnel and equipment can be accurately monitored and tracked.

Description

Technical Field

[0001] The present invention relates to the technical field of personnel positioning, and in particular to a mine full-coverage positioning method and system based on UWB and video blind spot completion. Background Art

[0002] Video surveillance and UWB positioning systems have become a necessity for underground mines. Video surveillance technology can monitor the mine environment and the activities of people in the monitoring screen in real time, but the monitoring range is very limited. The UWB positioning system can accurately obtain the location information of personnel, which can provide an important basis for the formulation of emergency evacuation plans. However, due to the complex terrain and harsh environment of underground mines, if UWB positioning signals are to fully cover every corner of the mine, a large number of positioning base stations need to be deployed, which requires a lot of investment, especially for some smaller chambers. If positioning is to be achieved, base stations also need to be deployed, which has a high cost, and the benefits of investment and actual output are not equal.

[0003] Therefore, it is necessary to provide an integrated monitoring and positioning data system to realize the dynamic display of personnel location information based on virtual reality technology. Summary of the invention

[0004] According to the technical problems raised above, a mine full coverage positioning method and system based on UWB and video blind spot completion is provided. The present invention deploys video monitoring equipment in the positioning blind area, and develops an auxiliary positioning method based on deep learning based on video stream data, and establishes a corresponding platform to achieve integrated management and control of monitoring and positioning data, and at the same time realize dynamic display of personnel location information based on virtual reality technology. In this way, not only can the coverage and positioning accuracy of the monitoring system be improved, but also the intelligent management and application of monitoring data can be realized, providing more reliable technical support for mine safety production.

[0005] The technical means adopted by the present invention are as follows:

[0006] A mine full coverage positioning method based on UWB and video blind spot completion, comprising:

[0007] Determine key monitoring areas and blind spots, deploy surveillance cameras, and adjust the angles of surveillance cameras;

[0008] Collect employee photos and personal information, use YOLO to build a personnel recognition model, and perform lightweight processing;

[0009] Using the ranging algorithm based on the principle of monocular imaging, the location information of the person is calculated according to the recognition results of the person recognition model;

[0010] Edge computing methods are used to dynamically calculate the real-time information of personnel locations, and the personnel locations are deduced and visualized in real time.

[0011] Furthermore, a deployment plan is designed according to the key monitoring areas and blind spot locations, monitoring cameras are installed according to the deployment plan, and UWB positioning base stations are deployed in the transport lanes to configure network information.

[0012] Furthermore, the personnel recognition model is constructed using YOLO, the collected employee photos and personal information are integrated into an initial network model, the personnel information is represented in the form of a mask, the initial network model is converted into an intermediate representation format using the tools ONNX-TensorFlow and OPENVINO, and OPENVINO is used to simplify the floating point precision of the network model.

[0013] Furthermore, the distance measurement algorithm based on the monocular imaging principle is used to calculate the position information of the person according to the recognition result of the person recognition model, which specifically includes:

[0014] Use deep learning technology to estimate the depth of the person image captured by the surveillance camera and obtain the distance information between the person and the surveillance camera; use the angle measurement function of the surveillance camera to obtain the inclination angle between the surveillance camera and the ground; according to the installation height and focal length parameters of the surveillance camera, combined with the angle measurement results, calculate the real size of the surveillance person image through the similar triangle method;

[0015] By analyzing the size changes of the person's image in the field of view of the surveillance camera and combining it with the size information of known environmental markers, a proportional calculation is performed to calculate the actual distance between the person and the camera. Based on the known camera installation height, angle information and the distance between the user and the camera, the triangulation method is used to calculate the specific location information of the person.

[0016] Furthermore, the edge computing method is used to realize real-time detection of personnel information and dynamic calculation of location information, and to supplement the positioning results of the blind spots of UWB positioning base stations.

[0017] Furthermore, the visualization includes: combining the personnel image and the personnel's specific location information with the virtual scene of the mining area to realize the visualization of the three-dimensional personnel activity path in the virtual scene.

[0018] Corresponding to the mine full coverage positioning method based on UWB and video blind spot completion in the present invention, the present invention also provides a mine full coverage positioning system based on UWB and video blind spot completion, including: a positioning information acquisition module, a personnel identification module, a position information calculation module and a three-dimensional visualization module, wherein:

[0019] The positioning information acquisition module is used to determine the key monitoring area and the blind spot position, and to deploy monitoring cameras and adjust the angle of the monitoring cameras;

[0020] The personnel identification module is used to collect employee photos and personal information, build a personnel identification model using YOLO, and perform lightweight processing;

[0021] The position information calculation module is used to calculate the position information of the person according to the recognition result of the person recognition model by using a distance measurement algorithm based on the monocular imaging principle;

[0022] The three-dimensional visualization module is used to dynamically calculate the real-time information of personnel positions by edge computing methods, and to perform real-time deduction and visualization of personnel positions.

[0023] Compared with the prior art, the present invention has the following advantages:

[0024] The mine full coverage positioning method and system based on UWB and video blind spot completion provided by the present invention determines the key monitoring area and the blind spot position, and deploys monitoring cameras, and adjusts the angle of the monitoring cameras; collects employee photos and personal information, uses YOLO to build a personnel recognition model, and performs lightweight processing; uses a ranging algorithm based on the principle of monocular imaging, and calculates the position information of the personnel according to the recognition result of the personnel recognition model; uses edge computing methods to dynamically calculate the real-time information of the personnel position, and performs real-time deduction and visualization of the personnel position. The present invention realizes the full range positioning of the mining area, ensuring that the position of personnel and equipment can be accurately monitored and tracked regardless of the environment; integrates the video and positioning system into one platform, and intuitively displays the personnel position information through virtual reality technology, eliminating the data islands between monitoring and positioning data, and improving management efficiency and positioning accuracy.

[0025] Based on the above reasons, the present invention can be widely promoted in the fields of personnel positioning and the like. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0027] Figure 1 This is a flow chart of the mine full coverage positioning method based on UWB and video blind spot completion in the present invention.

[0028] Figure 2 This is a layout design diagram for an embodiment of the present invention.

[0029] Figure 3 This is the interface of the mine full coverage positioning system based on UWB and video blind spot completion in the embodiment of the present invention.

[0030] Figure 4 This is a positioning card list in an embodiment of the present invention.

[0031] Figure 5 This is the personnel history trajectory backtracking function interface in the embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] like Figure 1 As shown, the present invention provides a mine full coverage positioning method based on UWB and video blind spot completion, comprising:

[0035] Determine key monitoring areas and blind spots, deploy surveillance cameras, and adjust the angles of surveillance cameras;

[0036] During specific implementation, as a preferred embodiment of the present invention, a deployment plan is designed according to key monitoring areas and blind spot locations, monitoring cameras are installed according to the deployment plan, and UWB positioning base stations are deployed in transport lanes, network information is configured, and a Socket service for transmitting personnel positioning information is formed.

[0037] During implementation, an on-site survey of the air-raid shelter was conducted, and a UWB positioning base station deployment plan was designed with low cost and high coverage as the goal. The deployment plan is as follows: Figure 2 As shown, UWB positioning base stations are deployed in the main transport tunnels. In small chambers, the positioning signal is poor, but considering the cost issue, the base station is not installed here. Instead, the location information of the personnel is determined by using cameras.

[0038] Collect employee photos and personal information, use YOLO to build a personnel recognition model, and perform lightweight processing;

[0039] In specific implementation, as a preferred embodiment of the present invention, YOLO is used to build a personnel recognition model, and the collected employee photos and personal information are integrated into an initial network model in *.pt format to represent the personnel information in the form of a mask;

[0040] In order to improve the portability and timeliness of the model, the YOLO model is converted into the *.onnx format using tools such as ONNX-TensorFlow, and further converted into an intermediate representation format (IR) suitable for hardware inference using OPENVINO. This process generates a set of files including the weight file .bin (containing binary data of weights and biases), the structure file .xml (containing information about the network topology), and the configuration file .json (data exchange format). The IR model, as an intermediate format, can run under different frameworks and flexibly switch to different formats to increase the computing speed, ensuring high performance and flexible configuration when the model is deployed. Use OPENVINO to simplify the floating-point precision of the model from FP32 to FP16 or even FP8 to improve the timeliness of personnel information detection;

[0041] Using the ranging algorithm based on the principle of monocular imaging, the location information of the person is calculated according to the recognition results of the person recognition model;

[0042] In specific implementation, as a preferred embodiment of the present invention, a ranging algorithm based on the monocular imaging principle is used to calculate the position information of a person according to the recognition result of a person recognition model, which specifically includes:

[0043] Use deep learning technology to estimate the depth of the person image captured by the surveillance camera and obtain the distance information between the person and the surveillance camera; use the angle measurement function of the surveillance camera to obtain the inclination angle between the surveillance camera and the ground; according to the installation height and focal length parameters of the surveillance camera, combined with the angle measurement results, calculate the real size of the surveillance person image through the similar triangle method;

[0044] By analyzing the size changes of the person's image in the field of view of the surveillance camera and combining it with the size information of known environmental markers, such as markers or reference objects on the ground, a proportional calculation is performed to calculate the actual distance between the person and the camera. Based on the known camera installation height, angle information and the distance between the user and the camera, the triangulation method is used to calculate the specific location information of the person.

[0045] Edge computing methods are used to dynamically calculate the real-time information of personnel locations, and the personnel locations are deduced and visualized in real time.

[0046] During implementation, configure the IP information of each camera and deploy the edge computing server.

[0047] In specific implementation, as a preferred embodiment of the present invention, the edge computing method is used to realize the real-time detection of personnel information and the dynamic calculation of location information, supplement the positioning results of the blind area of ​​the UWB positioning base station, and realize the acquisition of full-range location information under low-cost conditions.

[0048] Visualization includes: combining personnel images and their specific location information with the virtual scene of the mining area to achieve visualization of three-dimensional personnel activity paths in the virtual scene.

[0049] Corresponding to the mine full coverage positioning method based on UWB and video blind spot completion in the present invention, the present invention also provides a mine full coverage positioning system based on UWB and video blind spot completion, including: a positioning information acquisition module, a personnel identification module, a position information calculation module and a three-dimensional visualization module, wherein:

[0050] The present invention utilizes Vue to develop the front end, Java to develop the back end, MySQL to persist data, and Unity3D to build a mine virtual reality scene, thereby establishing a mine full coverage positioning platform based on UWB and video blind spot completion.

[0051] The positioning information collection module is used to determine the key monitoring areas and blind spots, deploy monitoring cameras, and adjust the angles of the monitoring cameras; establish a Socket server that matches the UWB positioning base station.

[0052] The personnel identification module is used to collect employee photos and personal information, build a personnel identification model using YOLO, and perform lightweight processing; during implementation, the personnel information is bound to the positioning card to achieve the correlation between the positioning UWB positioning information and the personnel, such as Figure 3 As shown in the figure, users can view the existing location card information and the personnel information that matches it, and can also modify the binding relationship between the location card and the personnel through the system, such as Figure 4 shown.

[0053] The location information calculation module is used to calculate the location information of personnel according to the recognition results of the personnel identification model using a ranging algorithm based on the principle of monocular imaging; through the GB / T28181 protocol, it accesses the original video stream and the results of personnel identification and location deduction, and receives and interprets the personnel location information in real time in combination with the relationship between personnel and positioning cards. On the one hand, the interpretation results are stored persistently in the cloud to facilitate the query of historical activity trajectories; on the other hand, the data is further transmitted to the three-dimensional visualization display module of personnel location information driven by UWB-monitoring information through the Socket protocol to display the location and activity information of personnel in a virtual reality way. The HTTP protocol is used to access the personnel location information dynamically calculated by edge computing, and store it in the data for persistent storage to facilitate the query of historical activity trajectories;

[0054] The 3D visualization module is used to dynamically calculate the real-time information of personnel positions using edge computing methods, and to perform real-time deduction and visualization of personnel positions. Based on Unity3D, a virtual reality scene consistent with the mine is established, such as Figure 3 As shown in the figure, the functional relationship between the positioning information driven by UWB and surveillance video data and the world coordinates of the virtual reality scene is established respectively. While the positioning information is persisted, the real-time positioning information is converted into the world coordinates of Unity3D according to the above functional relationship, and the Socket communication relationship between the two is further established, so as to realize the real-time display of the positioning information in the virtual reality scene. At the same time, the historical positioning information in the database is obtained through the HTTP protocol, and the historical activity trajectory of the personnel is calculated based on the above functional relationship. The user can visualize the activity path of the personnel in the virtual reality scene, as shown in FIG. Figure 5 shown.

[0055] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0056] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0057] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0058] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0059] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0060] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A mine full coverage positioning method based on UWB and video blind spot completion, characterized in that: include: Determine key monitoring areas and blind spots, deploy surveillance cameras, and adjust the angles of surveillance cameras; Collect employee photos and personal information, use YOLO to build a personnel recognition model, and perform lightweight processing; Using the ranging algorithm based on the principle of monocular imaging, the location information of the person is calculated according to the recognition results of the person recognition model; Edge computing methods are used to dynamically calculate the real-time information of personnel locations, and the personnel locations are deduced and visualized in real time.

2. The mine full coverage positioning method based on UWB and video blind spot completion according to claim 1 is characterized in that: Design a deployment plan based on the key monitoring areas and blind spot locations, install monitoring cameras according to the deployment plan, deploy UWB positioning base stations in the transport lanes, and configure network information.

3. The mine full coverage positioning method based on UWB and video blind spot completion according to claim 1 is characterized in that: The method uses YOLO to build a personnel recognition model, integrates the collected employee photos and personal information into an initial network model, represents the personnel information in the form of a mask, uses the tools ONNX-TensorFlow and OPENVINO to convert the initial network model into an intermediate representation format, and uses OPENVINO to simplify the floating-point precision of the network model.

4. The mine full coverage positioning method based on UWB and video blind spot completion according to claim 1 is characterized in that: The method of calculating the position information of a person by using a ranging algorithm based on the principle of monocular imaging according to the recognition result of a person recognition model specifically includes: Use deep learning technology to estimate the depth of the person image captured by the surveillance camera and obtain the distance information between the person and the surveillance camera; use the angle measurement function of the surveillance camera to obtain the inclination angle between the surveillance camera and the ground; according to the installation height and focal length parameters of the surveillance camera, combined with the angle measurement results, calculate the real size of the surveillance person image through the similar triangle method; By analyzing the size changes of the person's image in the field of view of the surveillance camera and combining it with the size information of known environmental markers, a proportional calculation is performed to calculate the actual distance between the person and the camera. Based on the known camera installation height, angle information and the distance between the user and the camera, the triangulation method is used to calculate the specific location information of the person.

5. The mine full coverage positioning method based on UWB and video blind spot completion according to claim 1 is characterized in that: The edge computing method is used to realize real-time detection of personnel information and dynamic calculation of location information, and to supplement the positioning results of the blind spots of UWB positioning base stations.

6. The mine full coverage positioning method based on UWB and video blind spot completion according to claim 1 is characterized in that: The visualization includes: combining the personnel image and the personnel's specific location information with the virtual scene of the mining area to realize the visualization of the three-dimensional personnel activity path in the virtual scene.

7. A mine full coverage positioning system based on UWB and video blind spot completion implemented based on the mine full coverage positioning method based on UWB and video blind spot completion as described in any one of claims 1 to 6, characterized in that: include: Positioning information acquisition module, personnel identification module, position information calculation module and three-dimensional visualization module, among which: The positioning information acquisition module is used to determine the key monitoring area and the blind spot position, and to deploy monitoring cameras and adjust the angle of the monitoring cameras; The personnel identification module is used to collect employee photos and personal information, build a personnel identification model using YOLO, and perform lightweight processing; The position information calculation module is used to calculate the position information of the person according to the recognition result of the person recognition model by using a distance measurement algorithm based on the monocular imaging principle; The three-dimensional visualization module is used to dynamically calculate the real-time information of personnel positions by edge computing methods, and to perform real-time deduction and visualization of personnel positions.

Images