Underground coal face visual target positioning method and device, terminal and storage medium

Abstract

The invention provides an underground coal face visual target positioning method and device, a terminal and a storage medium, and the method comprises the steps: recognizing each target object in an initial frame image of a panoramic video of each coal face, and tracking each target object, obtaining a hydraulic support detection frame position corresponding to the hydraulic support and a target detection frame position corresponding to the to-be-positioned object in each video image collected by each panoramic camera; determining a matching relationship between the hydraulic support detection frame position and the hydraulic support number in each video image, and determining a target hydraulic support detection frame position closest to the target detection frame position; and searching the matching relationship to obtain a target hydraulic support number corresponding to the position of the target hydraulic support detection frame, and recording the target hydraulic support number as a positioning result of the to-be-positioned object. According to the invention, positioning is carried out in a mode of shooting videos by the panoramic camera, the method is not easily influenced by severe environments in a mine, the anti-interference capability is strong, and the positioning accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of target positioning technology, and in particular to a method, device, terminal and storage medium for visual target positioning at an underground coal mining face. Background Art

[0002] In recent years, the demand for intelligent underground operations in coal mining enterprises has been increasing. In the process of advancing the comprehensive mining face of coal mines, the comprehensive mining equipment of coal mines cooperates in coal mining, and it is necessary to monitor, track and locate the coal mining machine, hydraulic support and personnel in real time, so that ground supervisors can better grasp the dynamic situation of underground workers and take corresponding measures in emergency situations to improve the safety of underground operations. The core technology of the positioning system is to provide real-time communication functions for underground personnel. The communication equipment underground has gone through the stage from wired communication to wireless communication. With the rapid development of information technology, wireless communication will gradually replace wired communication. At present, the underground positioning systems of mines that commonly use wireless communication methods mainly include RFID positioning systems, WIFI positioning systems and GIS positioning systems, which can achieve strong information interaction functions.

[0003] The mine operating environment is complex and dangerous. Although the traditional underground coal mine positioning system has been able to achieve strong information interaction functions, it still has defects in actual application, including channel fading, weak penetration, and weak anti-interference ability. The traditional positioning system requires the use of specific sensors such as laser radar and infrared sensors. The harsh environment in the mine includes high humidity, dust, vibration, and chemical corrosion. These factors not only increase the difficulty of equipment installation, but also easily cause equipment damage, and weak anti-interference ability, which affects its long-term stable operation, and then leads to inaccurate positioning results.

[0004] Therefore, the prior art has defects and needs to be improved and developed. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a method, device, terminal and storage medium for visual target positioning of an underground coal mining face are provided, aiming to solve the problem that sensors are needed for positioning in the prior art, and the harsh environment in the mine easily affects the sensors, thereby causing inaccurate positioning.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0007] A method for visual target positioning at an underground coal mining face, wherein the method comprises:

[0008] Acquire a panoramic video of a coal mining face collected by a plurality of panoramic cameras in real time, and identify each target object in an initial frame image of each of the panoramic videos of the coal mining face, wherein the target object includes a plurality of hydraulic supports and an object to be positioned;

[0009] Tracking each of the target objects based on the panoramic video of the coal mining face to obtain the hydraulic support detection frame position corresponding to the hydraulic support and the target detection frame position corresponding to the object to be located in each video image captured by each panoramic camera;

[0010] Determine the matching relationship between the hydraulic support detection frame position and the hydraulic support number in each video image, and determine the target hydraulic support detection frame position that is closest to the target detection frame position;

[0011] The matching relationship is searched to obtain the target hydraulic support number corresponding to the position of the target hydraulic support detection frame, and the target hydraulic support number is recorded as the positioning result of the object to be positioned.

[0012] In one embodiment of the present application, identifying each target object in the initial frame image of each coal mining face panoramic video includes:

[0013] The rotating detection frame of the YOLOv8-OBB target detection algorithm is used to perform target detection and recognition on the initial frame image of the coal mining face panoramic video to obtain various target objects.

[0014] In one embodiment of the present application, each target object is tracked based on the panoramic video of the coal mining face to obtain the hydraulic support detection frame position corresponding to the hydraulic support and the target detection frame position corresponding to the object to be located in each video image captured by each panoramic camera, including:

[0015] Based on the panoramic video of the coal mining face, each of the target objects is continuously captured in time;

[0016] Use the YOLOv8-Track system to continuously track the same target object in the time domain to obtain the detection frame position of each target object, the confidence corresponding to the detection frame, the category corresponding to each detection frame, and the ID of the tracked object;

[0017] According to the detection frame position of each target object, the confidence corresponding to the detection frame, the category corresponding to each detection frame and the ID of the tracked object, the hydraulic support detection frame position corresponding to each hydraulic support and the target detection frame position corresponding to the object to be located are obtained.

[0018] In one embodiment of the present application, each of the cameras is arranged on a hydraulic support;

[0019] Determine the matching relationship between the hydraulic support detection frame position and the hydraulic support number in each video image, including:

[0020] Obtain the image center point coordinates of each video image, and calculate the hydraulic support detection frame position closest to the image center point coordinates in each video image;

[0021] Determine the source camera of the video image, and determine the hydraulic support detection frame position closest to the coordinates of the center point of the image as the reference hydraulic support detection frame position corresponding to the hydraulic support where the source camera is located;

[0022] Obtaining a pre-stored correspondence between a camera number and a hydraulic support number, wherein the correspondence is determined based on the hydraulic support where the camera is located;

[0023] Searching for the corresponding relationship, obtaining the reference hydraulic support number corresponding to the camera number of the source camera, and establishing a matching relationship between the reference hydraulic support detection frame position and the reference hydraulic support number;

[0024] According to the preset hydraulic support numbering rules and the reference hydraulic support number, the hydraulic support numbers corresponding to the other hydraulic support detection frame positions in the video image except the reference hydraulic support detection frame position are determined, and the matching relationship between all the hydraulic support detection frame positions and the hydraulic support numbers in the video image is obtained.

[0025] In one embodiment of the present application, determining the target hydraulic support detection frame position that is closest to the target detection frame position includes:

[0026] Obtaining the coordinates of the center points of the hydraulic supports of each hydraulic support based on the positions of the detection frames of each hydraulic support in the video image, and obtaining the coordinates of the target center point of the object to be positioned based on the position of the target detection frame;

[0027] The distances between the center point coordinates of each of the hydraulic supports and the center point coordinates of the target are calculated, and the target hydraulic support detection frame position closest to the target detection frame position is determined according to each of the distances.

[0028] In one embodiment of the present application, after searching the matching relationship, obtaining the target hydraulic support number corresponding to the position of the target hydraulic support detection frame, and recording the target hydraulic support number as the positioning result of the object to be positioned, the method further includes:

[0029] The positioning result of the object to be positioned is uploaded to the cloud for storage.

[0030] In one embodiment of the present application, a plurality of panoramic cameras are provided, and the plurality of panoramic cameras are provided on different hydraulic supports.

[0031] The present application also provides a visual target positioning device for an underground coal mining face, wherein the device comprises:

[0032] An identification module is used to obtain in real time a panoramic video of a coal mining face collected by a plurality of panoramic cameras, and to identify each target object in an initial frame image of each of the panoramic videos of the coal mining face, wherein the target object includes a plurality of hydraulic supports and an object to be positioned;

[0033] A tracking module, used to track each of the target objects based on the panoramic video of the coal mining face, and obtain the hydraulic support detection frame position corresponding to the hydraulic support and the target detection frame position corresponding to the object to be located in each video image captured by each panoramic camera;

[0034] A determination module, used to determine the matching relationship between the hydraulic support detection frame position and the hydraulic support number in each video image, and to determine the target hydraulic support detection frame position that is closest to the target detection frame position;

[0035] The recording module is used to search for the matching relationship, obtain the target hydraulic support number corresponding to the position of the target hydraulic support detection frame, and record the target hydraulic support number as the positioning result of the object to be positioned.

[0036] The present application also provides a terminal, which includes: a memory, a processor, and an underground coal mining face visual target positioning program stored in the memory and run on the processor, and the underground coal mining face visual target positioning program, when executed by the processor, implements the steps of the underground coal mining face visual target positioning method as described above.

[0037] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the underground coal mining face visual target positioning method as described above.

[0038] The present invention provides a method, device, terminal and storage medium for visual target positioning of an underground coal mining face, the method comprising: acquiring a panoramic video of a coal mining face collected by a plurality of panoramic cameras in real time, identifying each target object in an initial frame image of each of the panoramic videos of the coal mining face, the target object comprising a plurality of hydraulic supports and an object to be positioned; tracking each of the target objects based on the panoramic video of the coal mining face, obtaining a hydraulic support detection frame position corresponding to the hydraulic support in each video image collected by each panoramic camera and a target detection frame position corresponding to the object to be positioned; determining a matching relationship between the hydraulic support detection frame position and the hydraulic support number in each video image, and determining a target hydraulic support detection frame position that is closest to the target detection frame position; searching for the matching relationship, obtaining a target hydraulic support number corresponding to the target hydraulic support detection frame position, and recording the target hydraulic support number as a positioning result of the object to be positioned. The present invention utilizes the panoramic video of the coal mining face collected by a panoramic camera, tracks each target object based on the panoramic video of the coal mining face, and then obtains the positional relationship between each object to be positioned and the hydraulic support, and then locates the object to be positioned according to the number of the hydraulic support. The way the panoramic camera shoots video is not easily affected by the harsh environment in the mine, has strong anti-interference ability, and improves the accuracy of positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of a preferred embodiment of the underground coal mining face visual target positioning method of the present invention;

[0040] Figure 2 It is a logic block diagram of a preferred embodiment of the underground coal mining face visual target positioning method of the present invention;

[0041] Figure 3 It is a functional principle block diagram of a preferred embodiment of the underground coal mining face visual target positioning device of the present invention;

[0042] Figure 4 It is a functional principle block diagram of a preferred embodiment of the terminal in the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0044] Accurately transmitting and receiving signals in the complex environment of a mine is crucial for safe operations, requiring monitoring and positioning technology to collect and transmit data in real time to ensure the real-time response of the monitoring system. The embodiments of the present application can accurately track and locate coal mining machines, hydraulic supports, and underground personnel, ensuring the accuracy and real-time nature of their operating trajectories, and can greatly improve the safety and efficiency of mine operations. Through accurate positioning information, the working status of coal mining machines and hydraulic supports can be monitored in real time to prevent equipment failures and accidents. At the same time, accurate tracking of personnel can ensure rapid positioning and rescue in emergency situations, thereby improving personnel safety. In addition, the embodiments of the present application can also be combined with automation and intelligent systems to achieve automatic control of coal mining machines and hydraulic supports, reduce manual intervention, and improve operating efficiency. The embodiments of the present application can promote the transformation of mine operations to automation and intelligence, and make important contributions to the mining industry.

[0045] See also Figure 1 , Figure 1 Flowchart of the method for visually locating a target at an underground coal mining face in the present invention. Figure 1 As shown, the underground coal mining face visual target positioning method described in the embodiment of the present invention includes:

[0046] Step S100: Acquire in real time a panoramic video of a coal mining face collected by a plurality of panoramic cameras, and identify each target object in an initial frame image of each of the panoramic videos of the coal mining face, wherein the target object includes a plurality of hydraulic supports and an object to be positioned.

[0047] Specifically, the hydraulic support in the mine is an important coal mining equipment, which uses hydraulic power to achieve lifting, forward movement and other movements, and is mainly used for the support and control of the roof of the comprehensive mining face, as well as the movement of the working face equipment. Since multiple hydraulic supports need to be set up in the mine to support the comprehensive mining face, and the hydraulic supports are set up relatively densely, the present invention uses the hydraulic support as a reference to locate the object to be located, and the object to be located can be personnel and / or coal mining machines. In this way, according to the deployment method of the panoramic system in the comprehensive mining face of the coal mine, due to the obstruction of the support column, the panoramic system monitoring angle of view safety coverage is the space area in front of the column. The system can intelligently identify the personnel and equipment in the area and match it with the support position, so as to realize the real-time positioning of the working face coal mining machine and the working face personnel, and the positioning accuracy is accurate to the support number.

[0048] The panoramic camera is a low-latency panoramic camera. According to the low-latency panoramic camera installed on the hydraulic support, real-time video images are obtained for subsequent processing. The low-latency panoramic camera generates panoramic videos by correcting and splicing the videos collected by each camera, and then reduces the network transmission delay of the panoramic video by encoding the video.

[0049] The embodiment of the present application can quickly and accurately capture the dynamic changes of the coal mining face by acquiring the panoramic video of the coal mining face captured by the panoramic camera in real time and immediately identifying the target object (including the hydraulic support and the object to be positioned) in the initial frame.

[0050] In an embodiment of the present application, each target object is identified in the initial frame image of each of the coal mining face panoramic videos, including: using the rotating detection frame of the YOLOv8-OBB target detection algorithm to perform target detection and recognition on the initial frame image of the coal mining face panoramic video to obtain each target object.

[0051] Specifically, the initial targets are identified from the initial frame image of the acquired video image, including coal mining machines, personnel and hydraulic supports. The embodiment of the present application adopts the YOLOv8-OBB target detection algorithm, uses a rotating detection frame to perform target detection and recognition on the video image, and identifies the coal mining machine, personnel and hydraulic supports in the video image.

[0052] The YOLOv8-OBB target detection algorithm is a model for oriented target detection. It introduces the concept of oriented bounding box (OBB) based on YOLOv8 to more accurately locate and identify objects with a specific direction in the image. Oriented target detection uses rotated bounding boxes (OBBs), which can better adapt to the actual direction and shape of the object, thereby providing more accurate positioning information. OBB is a rectangular box that contains an object and can be rotated. In addition to the width, height, and center point coordinates of the traditional bounding box, it also contains a rotation angle parameter. This rotation angle enables the OBB to fit the actual shape and orientation of the object more closely, thereby improving the accuracy of detection.

[0053] The rotation detection frame introduced by YOLOv8-OBB in the embodiment of the present application can more accurately fit the actual shape and orientation of the target object, especially for equipment such as tilted or rotating hydraulic supports that may exist in the coal mining face, the OBB detection frame can provide more accurate detection results. In addition, it can accurately identify the target object and reduce the processing of invalid areas, thereby optimizing the utilization of computing resources and improving the overall detection efficiency.

[0054] like Figure 1 As shown, the underground coal mining face visual target positioning method described in this embodiment also includes:

[0055] Step S200: Track each of the target objects based on the panoramic video of the coal mining face to obtain the hydraulic support detection frame position corresponding to the hydraulic support and the target detection frame position corresponding to the object to be located in each video image captured by each panoramic camera.

[0056] In the embodiment of the present application, the step S200 specifically includes:

[0057] Step S210: Based on the panoramic video of the coal mining face, each target object is continuously captured in time;

[0058] Step S220: Use the YOLOv8-Track system to continuously track the same target object in the time domain to obtain the detection frame position of each target object, the confidence corresponding to the detection frame, the category corresponding to each detection frame, and the ID of the tracked object;

[0059] Step S230, according to the detection frame position of each target object, the confidence corresponding to the detection frame, the category corresponding to each detection frame and the ID of the tracking object, the hydraulic support detection frame position corresponding to each hydraulic support and the target detection frame position corresponding to the object to be located are obtained.

[0060] Specifically, tracking is the continuous capture of coal mining machines, personnel and hydraulic supports in time, and the continuous tracking of the same target object in the time domain. The target tracking part is based on the target recognition stage, using YOLOv8-Track to track coal mining machines, personnel and hydraulic supports, and outputs the position of each target detection frame, the confidence corresponding to the detection frame, the category corresponding to each detection frame, and the ID of the tracked object. The category information corresponding to each detection frame helps identify the type of target object, such as hydraulic supports, objects to be located, etc. The ID of the tracked object assigns a unique identifier to each target object, so that different target objects can be accurately distinguished and tracked in consecutive frames.

[0061] YOLOv8-Track is a comprehensive system that combines the YOLOv8 target detection algorithm and multi-target tracking technology. It is mainly used to detect and track multiple targets in real time in video or image sequences.

[0062] The embodiment of the present application ensures full attention to the target object by continuously capturing each target object in the panoramic video of the coal mining face in time. The YOLOv8-Track system can continuously track the same target object in the time domain, effectively reducing the situation of target loss or mistracking, and improving the stability and continuity of target tracking.

[0063] like Figure 1 As shown, the underground coal mining face visual target positioning method described in this embodiment also includes:

[0064] Step S300, determining the matching relationship between the hydraulic support detection frame position and the hydraulic support number in each video image, and determining the target hydraulic support detection frame position that is closest to the target detection frame position.

[0065] In the embodiment of the present application, each of the cameras is arranged on the hydraulic support; and the step of determining the matching relationship between the hydraulic support detection frame position and the hydraulic support number in each video image comprises:

[0066] Obtain the image center point coordinates of each video image, and calculate the hydraulic support detection frame position closest to the image center point coordinates in each video image;

[0067] Determine the source camera of the video image, and determine the hydraulic support detection frame position closest to the coordinates of the center point of the image as the reference hydraulic support detection frame position corresponding to the hydraulic support where the source camera is located;

[0068] Obtaining a pre-stored correspondence between a camera number and a hydraulic support number, wherein the correspondence is determined based on the hydraulic support where the camera is located;

[0069] Searching for the corresponding relationship, obtaining the reference hydraulic support number corresponding to the camera number of the source camera, and establishing a matching relationship between the reference hydraulic support detection frame position and the reference hydraulic support number;

[0070] According to the preset hydraulic support numbering rules and the reference hydraulic support number, the hydraulic support numbers corresponding to the other hydraulic support detection frame positions in the video image except the reference hydraulic support detection frame position are determined, and the matching relationship between all the hydraulic support detection frame positions and the hydraulic support numbers in the video image is obtained.

[0071] Specifically, the center point of the video image in the client's visual range is calculated to obtain the coordinates of the center point of the image, and the center coordinates of the hydraulic support frame and the coordinates of the center point of the image are calculated to obtain the minimum distance. At this time, the position of the hydraulic support detection frame closest to the center point of the image (x M ,y M ), which is the hydraulic support where the current camera number is M, and it can be known that the hydraulic support number is M. Then the hydraulic supports are sorted in ascending order according to the horizontal coordinate values ​​of the center points of their detection frames. i ,y i ), when x i =x m , the hydraulic support is numbered M; when x i <x m , M decreases by 1 successively, when x i >x m , M is increased by 1 successively to obtain the matching result of the hydraulic support coordinates and its number.

[0072] This is because when a camera captures a video image, since the camera is located at the center of the image, the hydraulic support where the camera is located must also be at the center of the image. Therefore, the hydraulic support detection frame position closest to the image center coordinates is the hydraulic support where the camera is located, and then the number of the hydraulic support is obtained. The number of the hydraulic support where the camera is located is used as the reference hydraulic support number, and the position of the hydraulic support detection frame where the camera is located is used as the reference position, and then the numbers of other hydraulic supports in the video image are determined.

[0073] In the embodiment of the present application, determining the target hydraulic support detection frame position that is closest to the target detection frame position includes:

[0074] Obtaining the coordinates of the center points of the hydraulic supports of each hydraulic support based on the positions of the detection frames of each hydraulic support in the video image, and obtaining the coordinates of the target center point of the object to be positioned based on the position of the target detection frame;

[0075] The distances between the center point coordinates of each of the hydraulic supports and the center point coordinates of the target are calculated, and the target hydraulic support detection frame position closest to the target detection frame position is determined according to each of the distances.

[0076] The embodiment of the present application can accurately represent the positions of the hydraulic support and the object to be positioned in two-dimensional space by extracting the center point coordinates through the detection frame position information. Based on the center point coordinates, the distance between them can be calculated through simple geometric operations, which is simple and efficient and can quickly obtain results. Moreover, compared with other matching methods, the distance-based method is less susceptible to environmental noise, lighting changes or occlusion, thereby reducing the possibility of misjudgment.

[0077] like Figure 1 As shown, the underground coal mining face visual target positioning method described in this embodiment also includes:

[0078] Step S400: search for the matching relationship, obtain the target hydraulic support number corresponding to the target hydraulic support detection frame position, and record the target hydraulic support number as the positioning result of the object to be positioned.

[0079] The embodiment of the present application utilizes the positional relationship between the coal mining machine, personnel and hydraulic supports for positioning, and matches the positional relationship between the coal mining machine and the hydraulic supports, and the positional relationship between the personnel and the hydraulic supports. Each hydraulic support in the embodiment of the present application is numbered. In a specific embodiment, a low-latency panoramic camera is installed on the hydraulic supports, and a low-latency panoramic camera is installed every two hydraulic supports. Based on the obtained position information, category and ID of the target detection frame, the positional relationship between the personnel and the hydraulic supports and the positional relationship between the coal mining machine and the hydraulic supports are positioned.

[0080] Specifically, Figure 2 As shown, the distance between the center point of the personnel detection frame and the center point of each hydraulic support boundary frame is calculated. The nearest hydraulic support is determined, and its number is used as the hydraulic support number Y at the personnel location. The personnel position is output as the hydraulic support number Y at this time, that is, the personnel positioning is achieved. The distance between the center point of the coal mining machine detection frame in the current frame and the center point of each hydraulic support boundary frame is calculated, and the nearest hydraulic support is determined. Its number is used as the hydraulic support number X at the location of the coal mining machine, that is, the coal mining machine is accurately positioned. And the hydraulic support numbers X and Y can be displayed on the client.

[0081] In the embodiment of the present application, since the number is unique, in an emergency situation (such as equipment failure, accident rescue, etc.), the problem can be quickly located and handled accordingly by quickly searching and recording the position information of the numbered hydraulic support.

[0082] In one embodiment of the present application, after step S400, the method further includes: uploading the positioning result of the object to be positioned to the cloud for storage.

[0083] Specifically, the positioning results of the object to be positioned are synchronously uploaded to the cloud to realize cloud data storage. While realizing accurate positioning of the position information of personnel and hydraulic supports, and the coal mining machine and hydraulic supports, the information is stored in the cloud to realize information interaction between the well and the underground.

[0084] The embodiment of the present application provides a centralized data storage platform that can centrally manage all relevant positioning result data, which helps to achieve unified management and maintenance of data and avoid the risk of data dispersion and loss. In addition, cloud storage supports access from multiple devices and platforms, including computers, mobile phones, tablets, etc., allowing users to access positioning result data through any device at any time and any place.

[0085] In the embodiment of the present application, a plurality of panoramic cameras are provided, and the plurality of panoramic cameras are provided on different hydraulic supports.

[0086] Specifically, the embodiment of the present application uses a low-latency panoramic camera to perform intelligent monitoring of the underground coal mine comprehensive mining face, breaking the limitation of the single field of view of ordinary cameras and obtaining more complete underground comprehensive mining face information.

[0087] The embodiment of the present application can achieve all-round monitoring and coverage of the coal mining face by installing panoramic cameras on different hydraulic supports, which helps to eliminate monitoring blind spots and ensure that every corner can be clearly recorded.

[0088] In this way, in the mine environment, low-latency panoramic cameras are used to obtain real-time video images underground, and visual target detection, recognition and tracking algorithms are used to locate coal mining machines, personnel and hydraulic supports in the fully mechanized mining face of the coal mine, and the data is uploaded to the cloud in real time to ensure the safe, efficient and intelligent operation of the underground coal mining face. Cloud data storage provides an efficient, secure and reliable data management platform for underground coal mine operations, which helps to improve enterprise production efficiency, reduce costs, enhance operational safety, and support the digital transformation of enterprises.

[0089] In one embodiment, if Figure 3 As shown, based on the above-mentioned underground coal mining face visual target positioning method, the present invention also provides an underground coal mining face visual target positioning device, including:

[0090] The recognition module 100 is used to obtain the panoramic video of the coal mining face collected by a plurality of panoramic cameras in real time, and to recognize each target object in the initial frame image of each of the panoramic videos of the coal mining face, wherein the target object includes a plurality of hydraulic supports and an object to be positioned;

[0091] A tracking module 200 is used to track each of the target objects based on the panoramic video of the coal mining face, and obtain the hydraulic support detection frame position corresponding to the hydraulic support and the target detection frame position corresponding to the object to be located in each video image collected by each panoramic camera;

[0092] The determination module 300 is used to determine the matching relationship between the hydraulic support detection frame position and the hydraulic support number in each video image, and to determine the target hydraulic support detection frame position that is closest to the target detection frame position;

[0093] The recording module 400 is used to search for the matching relationship, obtain the target hydraulic support number corresponding to the position of the target hydraulic support detection frame, and record the target hydraulic support number as the positioning result of the object to be positioned.

[0094] Figure 4 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may include:

[0095] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0096] When the processor 502 executes the program, the underground coal mining face visual target positioning method provided in the above embodiment is implemented.

[0097] Furthermore, the terminal further includes:

[0098] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0099] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0100] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0101] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0102] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0103] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0104] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned method for visual target positioning of an underground coal mining face is implemented.

[0105] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0106] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0107] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes additional implementations, in which the order shown or discussed may not be followed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0108] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can read instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary and then storing it in a computer memory.

[0109] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0110] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0111] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0112] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

[0113] In summary, the present invention discloses a method, device, terminal and storage medium for visual target positioning of an underground coal mining face, the method comprising: acquiring a panoramic video of a coal mining face collected by a plurality of panoramic cameras in real time, identifying each target object in the initial frame image of each of the panoramic videos of the coal mining face, the target object comprising a plurality of hydraulic supports and an object to be positioned; tracking each of the target objects based on the panoramic video of the coal mining face, obtaining a hydraulic support detection frame position corresponding to the hydraulic support in each video image collected by each panoramic camera and a target detection frame position corresponding to the object to be positioned; determining a matching relationship between the hydraulic support detection frame position and the hydraulic support number in each video image, and determining a target hydraulic support detection frame position that is closest to the target detection frame position; searching for the matching relationship, obtaining a target hydraulic support number corresponding to the target hydraulic support detection frame position, and recording the target hydraulic support number as the positioning result of the object to be positioned. The present invention utilizes the panoramic video of the coal mining face collected by a panoramic camera, tracks each target object based on the panoramic video of the coal mining face, and then obtains the positional relationship between each object to be positioned and the hydraulic support, and then locates the object to be positioned according to the number of the hydraulic support. The way the panoramic camera shoots video is not easily affected by the harsh environment in the mine, has strong anti-interference ability, and improves the accuracy of positioning.

[0114] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for visual target positioning of an underground coal mining face, characterized in that: The method comprises: Acquire a panoramic video of a coal mining face collected by a plurality of panoramic cameras in real time, and identify each target object in an initial frame image of each of the panoramic videos of the coal mining face, wherein the target object includes a plurality of hydraulic supports and an object to be positioned; Tracking each of the target objects based on the panoramic video of the coal mining face to obtain the hydraulic support detection frame position corresponding to the hydraulic support and the target detection frame position corresponding to the object to be located in each video image captured by each panoramic camera; Determine the matching relationship between the hydraulic support detection frame position and the hydraulic support number in each video image, and determine the target hydraulic support detection frame position that is closest to the target detection frame position; The matching relationship is searched to obtain the target hydraulic support number corresponding to the position of the target hydraulic support detection frame, and the target hydraulic support number is recorded as the positioning result of the object to be positioned.

2. The method for visual target positioning of underground coal mining face according to claim 1, characterized in that: Identifying each target object in each initial frame image of the coal mining face panoramic video includes: The rotating detection frame of the YOLOv8-OBB target detection algorithm is used to perform target detection and recognition on the initial frame image of the coal mining face panoramic video to obtain various target objects.

3. The underground coal mining face visual target positioning method according to claim 1, characterized in that: Tracking each of the target objects based on the panoramic video of the coal mining face to obtain the hydraulic support detection frame position corresponding to the hydraulic support and the target detection frame position corresponding to the object to be positioned in each video image captured by each panoramic camera, including: Based on the panoramic video of the coal mining face, each of the target objects is continuously captured in time; Use the YOLOv8-Track system to continuously track the same target object in the time domain to obtain the detection frame position of each target object, the confidence corresponding to the detection frame, the category corresponding to each detection frame, and the ID of the tracked object; According to the detection frame position of each target object, the confidence corresponding to the detection frame, the category corresponding to each detection frame and the ID of the tracked object, the hydraulic support detection frame position corresponding to each hydraulic support and the target detection frame position corresponding to the object to be located are obtained.

4. The underground coal mining face visual target positioning method according to claim 1, characterized in that: Each of the cameras is arranged on a hydraulic support; Determine the matching relationship between the hydraulic support detection frame position and the hydraulic support number in each video image, including: Obtain the image center point coordinates of each video image, and calculate the hydraulic support detection frame position closest to the image center point coordinates in each video image; Determine the source camera of the video image, and determine the hydraulic support detection frame position closest to the coordinates of the center point of the image as the reference hydraulic support detection frame position corresponding to the hydraulic support where the source camera is located; Obtaining a pre-stored correspondence between a camera number and a hydraulic support number, wherein the correspondence is determined based on the hydraulic support where the camera is located; Searching for the corresponding relationship, obtaining the reference hydraulic support number corresponding to the camera number of the source camera, and establishing a matching relationship between the reference hydraulic support detection frame position and the reference hydraulic support number; According to the preset hydraulic support numbering rules and the reference hydraulic support number, the hydraulic support numbers corresponding to the other hydraulic support detection frame positions in the video image except the reference hydraulic support detection frame position are determined, and the matching relationship between all the hydraulic support detection frame positions and the hydraulic support numbers in the video image is obtained.

5. The method for visual target positioning of underground coal mining face according to claim 1, characterized in that: Determining the target detection frame position that is closest to the target hydraulic support detection frame position includes: Obtaining the coordinates of the center points of the hydraulic supports of each hydraulic support based on the positions of the detection frames of each hydraulic support in the video image, and obtaining the coordinates of the target center point of the object to be positioned based on the position of the target detection frame; The distances between the center point coordinates of each of the hydraulic supports and the center point coordinates of the target are calculated, and the target hydraulic support detection frame position closest to the target detection frame position is determined according to each of the distances.

6. The method for visual target positioning of underground coal mining face according to claim 1, characterized in that: After searching the matching relationship to obtain the target hydraulic support number corresponding to the target hydraulic support detection frame position and recording the target hydraulic support number as the positioning result of the object to be positioned, the method further includes: The positioning result of the object to be positioned is uploaded to the cloud for storage.

7. The method for visually locating an underground coal mining face according to claim 1, characterized in that: There are multiple panoramic cameras, and the multiple panoramic cameras are arranged on different hydraulic supports.

8. A visual target positioning device for underground coal mining face, characterized in that: The device comprises: An identification module is used to obtain in real time a panoramic video of a coal mining face collected by a plurality of panoramic cameras, and to identify each target object in an initial frame image of each of the panoramic videos of the coal mining face, wherein the target object includes a plurality of hydraulic supports and an object to be positioned; A tracking module, used to track each of the target objects based on the panoramic video of the coal mining face, and obtain the hydraulic support detection frame position corresponding to the hydraulic support and the target detection frame position corresponding to the object to be located in each video image captured by each panoramic camera; A determination module, used to determine the matching relationship between the hydraulic support detection frame position and the hydraulic support number in each video image, and to determine the target hydraulic support detection frame position that is closest to the target detection frame position; The recording module is used to search for the matching relationship, obtain the target hydraulic support number corresponding to the position of the target hydraulic support detection frame, and record the target hydraulic support number as the positioning result of the object to be positioned.

9. A terminal, characterized in that: include: A memory, a processor, and an underground coal mining face visual target positioning program stored in the memory and executable on the processor, wherein the underground coal mining face visual target positioning program, when executed by the processor, implements the steps of the underground coal mining face visual target positioning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program can be executed to implement the steps of the underground coal mining face visual target positioning method as described in any one of claims 1 to 7.

Images