Positioning and early warning method and device for coal mining face equipment

Abstract

The present disclosure provides a positioning early warning method and device for a coal mining face equipment, and relates to the field of face equipment positioning. The specific steps are as follows: collecting a detection image and determining a target shooting device, and determining the support serial number of the corresponding hydraulic support according to the serial number of the target shooting device; determining a first target detection frame and a second target detection frame, and determining a front roller, a rear roller and a traction direction; determining the first support serial number of the corresponding hydraulic support of the front roller according to the position of the front roller; determining the second support serial number of the corresponding hydraulic support of the extended state of the support plate according to the third target detection frame in the support plate detection image; and generating early warning information according to the first support serial number and the second support serial number. The present disclosure realizes real-time and accurate monitoring of the rollers of the coal mining machine and the support plates by positioning the front roller and the extended state of the support plate, and generates early warning information according to the distance between the two, thereby improving the safety of continuous production of the coal mining face.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of working face equipment positioning, and particularly relates to a positioning and early warning method and device for coal mining working face equipment. BACKGROUND

[0002] The coal mining working face is one of the key links of underground production of a coal mine. To build an intelligent coal mining working face and realize remote intelligent control of the equipment of the coal mining working face, the position of a coal mining machine in the coal mining working face must be monitored in real time and accurately, so as to timely find an abnormality, reasonably dispose, and ensure continuous and safe production.

[0003] In the related art, the existing technology needs to use a calibration target for positioning of the working face equipment, but the calibration target of the working face is more likely to be polluted by coal dust of the working face, has low reliability, and cannot realize real-time and accurate monitoring of the working face equipment, so that a safety hazard of continuous production of the coal mining working face cannot be timely found. SUMMARY

[0004] The present disclosure provides a positioning and early warning method and device for coal mining working face equipment, to at least solve the problem that a safety hazard of continuous production of the coal mining working face cannot be timely found in the related art. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a positioning and early warning method for coal mining working face equipment is provided, comprising:

[0006] A shooting device arranged on a driving hydraulic support collects a detection image, a shooting device corresponding to the detection image with a drum of a coal mining machine is determined as a target shooting device, and a support serial number of a corresponding hydraulic support is determined according to a serial number of the target shooting device, wherein the drum of the coal mining machine includes a front drum and a rear drum;

[0007] A first target detection frame of a first drum and a second target detection frame of a second drum in a target detection image of the target shooting device are acquired, the front drum and the rear drum are determined in the first drum and the second drum according to the first target detection frame and the second target detection frame, and a traction direction of the coal mining machine is determined;

[0008] A first support serial number of a hydraulic support corresponding to the front drum is determined according to a position of a target detection frame corresponding to the front drum in a corresponding target detection image;

[0009] A detection image shot by a shooting device located in the traction direction of the front drum is determined as a rib plate detection image, a rib plate in the rib plate detection image and a state of the rib plate are detected, wherein the state of the rib plate includes a folded state and an extended state;

[0010] According to the third target detection frame corresponding to the stretched state of the guard board in the detection image, the second support serial number of the hydraulic support corresponding to the stretched state of the guard board is determined;

[0011] According to the first support serial number and the second support serial number, pre-warning information is generated.

[0012] Optionally, the step of collecting the detection image by the internal camera of the shooting device arranged on the driving hydraulic support specifically comprises:

[0013] The pose of the internal camera in the shooting device is initialized, so that the internal camera faces the direction of the coal wall and shoots the detection image.

[0014] Optionally, the step of determining the shooting device corresponding to the detection image with the shearer drum as the target shooting device specifically comprises:

[0015] The shearer drum detection flag of the shooting device corresponding to the detection image with the shearer drum is set to 1;

[0016] The shearer drum detection flag of the shooting device corresponding to the detection image without the shearer drum is set to 0.

[0017] Optionally, the step of determining the support serial number of the corresponding hydraulic support according to the serial number of the target shooting device specifically comprises:

[0018] The serial number N of the target shooting device and the preset number M of hydraulic supports corresponding to the shooting device are obtained;

[0019] According to the support serial number of the hydraulic support corresponding to the serial number N of the target shooting device and the preset number M of hydraulic supports, the support serial number range of the hydraulic support corresponding to the serial number of the target shooting device is determined as [(N-1)×M+1, N×M], wherein the hydraulic support serial number is a positive integer.

[0020] Optionally, the step of determining the front drum and the rear drum in the first drum and the second drum according to the first target detection frame and the second target detection frame, and determining the traction direction of the shearer specifically comprises:

[0021] The first vertex coordinates of the vertexes in the first target detection frame in the target detection image are obtained, and the second vertex coordinates of the vertexes in the second target detection frame in the target detection image are obtained;

[0022] According to the first vertex coordinates, the coordinates of the first center point in the first target detection frame are determined, and according to the second vertex coordinates, the coordinates of the second center point in the second target detection frame are determined;

[0023] The front roller and the rear roller are determined by comparing the coordinates of the first center point and the second center point, respectively, in the first roller and the second roller.

[0024] The direction from the rear drum to the front drum is determined as the traction direction of the coal mining machine.

[0025] Optionally, the step of comparing the coordinates of the first center point and the second center point to determine the front roller and the rear roller in the first roller and the second roller specifically includes:

[0026] Subtract the ordinate of the first center point from the ordinate of the second center point to obtain the difference;

[0027] If the difference is greater than 0, then the first roller is determined to be the rear roller and the second roller is determined to be the front roller;

[0028] If the difference is less than 0, then the first roller is determined to be the front roller and the second roller is determined to be the rear roller.

[0029] Optionally, the step of determining the first support serial number of the hydraulic support corresponding to the front roller based on the position of the target detection box corresponding to the front roller in the corresponding target detection image specifically includes:

[0030] Obtain the x-coordinate of the center point of the target detection frame corresponding to the front roller. c And obtain the pixel width D of the target detection image in the camera axis direction of the shooting device;

[0031] At least one first coordinate interval is determined based on the pixel width D and the preset number of hydraulic supports M, wherein each first coordinate interval contains the following number of pixels.

[0032] Based on the x-coordinate of the center point of the target detection frame corresponding to the front roller c The first coordinate interval in which the front roller is located determines the first support sequence number of the hydraulic support corresponding to the front roller.

[0033] Optionally, the step of determining the second support sequence number of the hydraulic support corresponding to the extended side plate based on the third target detection box corresponding to the extended side plate in the side plate detection image specifically includes:

[0034] Obtain the serial number N of the target shooting device and the preset number A of the side protection plates corresponding to the shooting device;

[0035] Obtain the center point coordinates of the third target detection box, and obtain the pixel width D of the side panel detection image in the direction of the camera axis of the shooting device;

[0036] at least one second coordinate interval is determined according to the pixel width D and the preset number of side guards A, wherein each of the second coordinate intervals contains a number of pixels

[0037] A second support serial number of the hydraulic support corresponding to the stretched side guard is determined according to a second coordinate interval in which the abscissa of the center point coordinate of the third target detection frame is located.

[0038] Optionally, the step of generating a warning information according to the first support serial number and the second support serial number specifically includes:

[0039] An absolute value of a difference between the first support serial number and the second support serial number is taken as a distance parameter, and a warning distance interval is obtained.

[0040] If the distance parameter is in the warning distance interval, a warning signal is generated.

[0041] According to a second aspect of the embodiments of the present disclosure, a positioning and warning device of a coal mining face equipment is provided, which includes:

[0042] An image acquisition module is configured to capture a detection image, determine a target photographing device corresponding to the detection image with a shearer drum as the target photographing device, and determine a support serial number of a corresponding hydraulic support according to a serial number of the target photographing device, wherein the shearer drum includes a front drum and a rear drum.

[0043] A traction direction determination module is configured to obtain a first target detection frame of a first drum and a second target detection frame of a second drum in a target detection image of the target photographing device, and determine the front drum and the rear drum and a traction direction according to the first target detection frame and the second target detection frame in the first drum and the second drum.

[0044] A first support serial number determination module is configured to determine a first support serial number of a hydraulic support corresponding to the front drum according to a position of a target detection frame corresponding to the front drum in a corresponding target detection image.

[0045] A side guard detection module is configured to determine a detection image captured by a photographing device located in a traction direction of the front drum as a side guard detection image, and detect a side guard in the side guard detection image and a state of the side guard.

[0046] A second support serial number determination module is configured to determine a second support serial number of a hydraulic support corresponding to a stretched side guard according to a third target detection frame corresponding to the stretched side guard in the side guard detection image.

[0047] An early warning module is configured to generate early warning information based on the first support sequence number and the second support sequence number.

[0048] According to a third aspect of embodiments of the present disclosure, an electronic device is provided, comprising:

[0049] a processor;

[0050] a memory for storing processor-executable instructions;

[0051] The processor is configured to execute the instructions to implement the method according to any one of the first aspect.

[0052] According to a fourth aspect of embodiments of the present disclosure, a computer-readable storage medium is provided, when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the method according to any one of the first aspect.

[0053] According to a fifth aspect of embodiments of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the method according to any one of the first aspect.

[0054] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects:

[0055] The present disclosure realizes real-time and accurate monitoring of the shearer drum and the support plate by positioning the front drum and the support plate in the stretched state and generating early warning information according to the distance between the two, thereby improving the safety of continuous production of the coal mining face.

[0056] The device is simple and low in cost, and the scheme only uses one kind of shooting device to realize the cutting interference detection in the fully mechanized mining process.

[0057] The data is highly reliable. The previous positioning method of the coal mining machine is locally realized to position the state of the coal mining machine, and the concept of establishing a hydraulic support position sequence of the whole working face based on video data has not been established.

[0058] It is durable. Some of the prior art solutions paste a two-dimensional code on the coal mining machine, and the pasted positioning two-dimensional code is often contaminated by coal dust and soil, this scheme does not need to use a two-dimensional code, and the cutting sawtooth of the shearer drum will always remain bright with the production action of cutting coal.

[0059] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0060] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate an embodiment consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure without imposing on the disclosure any undue limitations.

[0061] Figure 1 is a flow chart of a positioning early warning method of a coal mining face equipment according to an exemplary embodiment.

[0062] Figure 2 is a structure diagram of an intelligent video system according to an exemplary embodiment.

[0063] Figure 3 is a shearer and hydraulic support positioning map of a coal mining face according to an exemplary embodiment.

[0064] Figure 4 is a shearer schematic diagram according to an exemplary embodiment.

[0065] Figure 5 is a shearer drum target detection schematic diagram according to an exemplary embodiment.

[0066] Figure 6 is a shooting equipment and hydraulic support positioning map according to an exemplary embodiment

[0067] Figure 7 is a hydraulic support support plate state target detection schematic diagram according to an exemplary embodiment.

[0068] Figure 8 is a block diagram of a positioning early warning device of a coal mining face equipment according to an exemplary embodiment.

[0069] Figure 9 is a block diagram of a device according to an exemplary embodiment.

[0070] Figure 10 is a block diagram of a device according to an exemplary embodiment. DETAILED DESCRIPTION

[0071] In order to make the ordinary people in the art better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings.

[0072] It should be noted that the terms "first", "second", and the like in the description and claims of the present disclosure and the foregoing drawings are used to distinguish between similar objects, and are not necessarily used to describe a particular sequential or chronological order. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0073] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) involved in the present disclosure is all information authorized by the user or authorized by all parties.

[0074] The coal mining face is one of the key links of underground production of coal mines. In order to build an intelligent coal mining face and realize remote intelligent control of the equipment in the coal mining face, the position of the coal mining machine in the coal mining face must be monitored in real time and accurately, so as to timely find out the abnormality, reasonably dispose, and ensure continuous and safe production.

[0075] A coal mining machine positioning monitoring system is provided in the related technology, comprising an intelligent positioning monitoring device, a positioning target, and an upper computer. The intelligent positioning monitoring device comprises a 2D laser radar, a digital camera, a 3-axis tilt sensor, and a measurement and control microcomputer. When the coal mining machine is walking on the scraper conveyor and is in different positions, the measurement and control microcomputer controls the 2D laser radar to scan and measure the distance of the lateral positioning target, and the longitudinal position coordinates and the heading angle of the coal mining machine in the working face can be determined by using the measurement result; the digital camera photographs the lateral positioning target, and the positioning target number and the displacement of the positioning target relative to the camera can be determined by using image recognition, so that the transverse position coordinates of the coal mining machine in the working face can be determined; the 3-axis tilt sensor monitors the pitch angle and the roll angle of the coal mining machine, and realizes the monitoring of the attitude of the coal mining machine.

[0076] However, the positioning of the working face equipment needs to use the calibration target, but the calibration target of the working face is more easily polluted by the coal dust of the working face, has low reliability, and cannot realize real-time and accurate monitoring of the working face equipment, so that the safety hidden danger of continuous production of the coal mining face cannot be found in time.

[0077] Figure 1 is a flow chart of a positioning and early warning method of a coal mining face equipment according to an exemplary embodiment, as shown in Figure 1 The method comprises the following steps.

[0078] Step 101, take a detection image, determine the target shooting device corresponding to the detection image with the shearer drum as the target shooting device, and determine the support sequence number of the corresponding hydraulic support according to the serial number of the target shooting device, wherein the shearer drum includes a front drum and a rear drum shearer.

[0079] The main purpose of the embodiment is to solve the problem of monitoring the interference state of the hydraulic support guard plate and the shearer drum in the process of fully mechanized coal mining equipment cooperation, so as to realize continuous and safe production of the fully mechanized working face. To solve this problem, the first premise is to determine the position of the shearer drum and the hydraulic support guard plate and the traction direction of the shearer, and then to determine the position of the shearer drum and the hydraulic support guard plate. In order to achieve this purpose, the problem of cutting interference state monitoring can be divided into three problems, one is the shearer tracking positioning problem of the whole working face, the second is the traction direction discrimination of the shearer, and the third is the positioning problem of the front drum and the extended state guard of the shearer. The solutions to these three problems will be described in the following three stages. Figure 2 is a structural diagram of an intelligent video system according to an exemplary embodiment. As shown in Figure 2 , the system includes a shearer positioning module, a shearer traction direction determination module, and a shearer drum and guard plate positioning module, which respectively realize the positioning of the shearer position, the shearer traction direction, the shearer drum position and the guard plate position.

[0080] Due to the wide-angle lens of the shooting device of the working face, the field of view of a camera can cover the left and right adjacent hydraulic supports. In order to realize the tracking of the shearer of the whole working face, it is not necessary to install a shooting device on each hydraulic support. Therefore, in order to reduce the waste of equipment resources, the number of interval supports between two shooting devices can be determined according to the angle coverage of the camera lens. Since the serial number of the shooting device can be obtained in real time, the support serial number of the hydraulic support photographed by the shooting device can be determined according to the serial number of the shooting device and the interval support number.

[0081] Step 102, obtain the first target detection box of the first drum and the second target detection box of the second drum in the target detection image of the target shooting device, and determine the front drum and rear drum shearer and the traction direction according to the first target detection box and the second target detection box in the first drum and the second drum.

[0082] In this embodiment, the sequence number of the shooting device where the shearer drum is located is determined by the target detection algorithm, so as to locate the support sequence number of the hydraulic support corresponding to the middle position of the shearer, and realize the positioning of the shearer. However, it is not possible to determine which of the left and right two shearer drums is in front and which is behind. In this stage, according to the characteristics of the front drum being higher and the rear drum being lower during the operation of the shearer, the front and rear relationship of the left and right drums is determined, so as to determine the traction direction of the shearer.

[0083] Step 103, determining the first support serial number of the hydraulic support corresponding to the front roller according to the position of the target detection frame corresponding to the front roller in the corresponding target detection image;

[0084] Step 104, determining the detection image of the guard board photographed by the shooting device located in the traction direction of the front roller as the guard board detection image, and detecting the guard board and the state of the guard board in the guard board detection image;

[0085] detecting the third target detection frame corresponding to the guard board in the guard board detection image by the visual detection technology, and detecting the state of the guard board, wherein the state of the guard board includes the folded state and the stretched state.

[0086] Whether the state of the guard board is reasonable, that is, whether the distance between the front roller and the guard board in the stretched state is outside the safe distance. Therefore, in addition to the positioning of the front roller of the coal mining machine, the stretched guard board also needs to be positioned, and the positioning of the stretched guard board requires the identification of the state of the guard board. Therefore, the state of the guard board is directly divided into two categories for detection here, which are the guard board folded state detection and the guard board stretched state detection.

[0087] The purpose of positioning the stretched guard board is also to position its position in the hydraulic support sequence, and the method is basically the same as that of determining the front roller of the coal mining machine. First, the position of the shooting device of the front roller identification mark position 1 needs to be determined, and whether there is a stretched guard board in the traction direction of the coal mining machine is searched in sequence, so as to determine the sequence number of the pan-tilt camera where the stretched guard board is located, and then determine the support serial number of the hydraulic support where the stretched guard board is located.

[0088] Figure 7 is a hydraulic support guard board state target detection schematic diagram according to an example embodiment. As shown in Figure 7 13 and 14 are the third target detection frame corresponding to the guard board, if the guard board is in the folded state, the corresponding third target detection frame is set to green; if the guard board is in the stretched state, the corresponding third target detection frame is set to red.

[0089] Step 105, determining the second support serial number of the hydraulic support corresponding to the stretched guard board according to the third target detection frame corresponding to the stretched guard board in the guard board detection image;

[0090] Step 106, generating a warning information according to the first support serial number and the second support serial number.

[0091] In the embodiment, the above two stages determine the position of the coal mining machine and the traction direction of the coal mining machine, and on this basis, the position of the coal mining machine can be known without traversing the detection image of the working face again. Only the video of the shooting device in the traction direction of the coal mining machine is retrieved according to the position of the coal mining machine, and the monitoring of the interference state between the hydraulic support guard board and the drum cutting of the coal mining machine can be realized.

[0092] After the sequence numbers of the hydraulic supports in which the front drum and the extended state guard board are located are obtained through the above steps, whether the distance between the front drum of the coal mining machine and the extended state guard board is outside the safe distance can be determined according to the first support sequence number and the second support sequence number, and a warning information is generated.

[0093] Optionally, the step 101 specifically comprises:

[0094] The pose of the internal camera in the shooting device is initialized, so that the internal camera faces the coal wall direction and shoots the detection image, wherein the shooting device is arranged on the hydraulic support.

[0095] Figure 3 A coal mining machine and hydraulic support positioning diagram of a coal mining working face is shown according to an example embodiment. As shown in Figure 3 The wide-angle lens of the shooting device of the working face is selected, and the field of view of a camera can cover the left and right adjacent hydraulic supports. In order to realize the tracking of the coal mining machine in the whole working face, it is not necessary to install a shooting device on each hydraulic support, and therefore, in order to reduce the waste of equipment resources.

[0096] Optionally, the step 101 of determining the shooting device corresponding to the detection image of the coal mining machine drum as the target shooting device specifically comprises:

[0097] In step 201a, the coal mining machine drum detection flag bit of the shooting device corresponding to the detection image of the coal mining machine drum is set to 1;

[0098] In step 201b, the coal mining machine drum detection flag bit of the shooting device corresponding to the detection image without the coal mining machine drum is set to 0.

[0099] In the embodiment, all shooting devices on the hydraulic supports of the working face at the current time are traversed, and whether there is a coal mining machine drum in each shooting device is detected through a target detection algorithm. The flag bit of the shooting device with the coal mining machine drum is set to 1, and the coal mining machine drum detection flag bit of the other cameras is still 0.

[0100] Optionally, the step 101 of determining the support sequence number of the corresponding hydraulic support according to the sequence number of the target shooting device specifically comprises:

[0101] In step 202, the serial number N of the target shooting device and the preset number M of hydraulic supports corresponding to the shooting device are obtained.

[0102] In step 203, the serial number range of the hydraulic support corresponding to the serial number of the target shooting device is determined according to the serial number of the hydraulic support corresponding to the serial number N of the target shooting device and the preset number M of hydraulic supports, the serial number of the hydraulic support on which the target shooting device is installed is [(N-1)×M+1+N×M] / 2, and the serial number of the hydraulic support is a positive integer.

[0103] In the embodiment, the interval number of the hydraulic supports between the two shooting devices can be determined according to the angle coverage range of the camera lens, that is, the preset number M of hydraulic supports. In a possible embodiment, the interval number of the hydraulic supports between the shooting devices is 3, and it can be known that the serial number of the hydraulic support on which the Nth target shooting device is installed is 3[(N-1)×3+1+N×3] / 2=2, the serial number range of the hydraulic support corresponding to the serial number of the Nth shooting device is [3×(N-1)+1, 3N], and the serial number of the hydraulic support on which the first camera is located is [1, 3]. The purpose of this stage is to determine the serial number of the hydraulic support on which the coal mining machine is located.

[0104] It should be noted that since the shooting device is installed towards the coal wall, the range that can be shot to the left and right of the shooting device is the same, that is, the number of the hydraulic supports that can be shot is the same, and the number of the hydraulic supports that can be shot by each shooting device is an odd number plus the hydraulic support on which the shooting device is installed.

[0105] It should be noted that since all the hydraulic supports need to be shot by the shooting device to monitor the entire coal mining face, the serial number of the hydraulic support on which the first shooting device is installed needs to be less than or equal to (1+M) / 2, and the serial number of the hydraulic support on which each adjacent shooting device is located needs to be different by M. In the embodiment, the serial number of the hydraulic support on which the first shooting device is installed is (1+3) / 2=2. If the serial number of the hydraulic support on which the first shooting device is installed is 1, the left and right limits of the serial number range of the hydraulic support corresponding to the serial number of the Nth shooting device need to be reduced by 1. The serial number range of the hydraulic support corresponding to the serial number of the target shooting device is deduced in the case of other values of M. Alternatively, the step of determining the front roller and the rear roller and determining the traction direction of the coal mining machine according to the first target detection frame and the second target detection frame in the first roller and the second roller includes the following steps.

[0106] Step 301, obtaining a first vertex coordinate of a vertex in the first target detection frame in the target detection image, and obtaining a second vertex coordinate of a vertex in the second target detection frame in the target detection image;

[0107] Step 302, determining a coordinate of a first center point in the first target detection frame according to the first vertex coordinate, and determining a coordinate of a second center point in the second target detection frame according to the second vertex coordinate;

[0108] In the embodiment, the first target detection frame and the second target detection frame are obtained by a target detection algorithm. The first target detection frame and the second target detection frame are rectangular frames, each having four vertices. The corner points of the diagonal lines of the first target detection frame and the second target detection frame are the corresponding center points.

[0109] Step 303, comparing the coordinate of the first center point with the coordinate of the second center point to determine the front drum and the rear drum in the first drum and the second drum.

[0110] Figure 4 is a schematic diagram of a coal mining machine according to an exemplary embodiment. As described above, the coal mining machine includes two coal mining machine drums 6 and 7, and the front drum is higher and the rear drum is lower during the coal mining process. If the coal mining machine drum 6 is higher and the coal mining machine drum 7 is lower during the coal mining process, the coal mining machine drum 6 is the front drum, and the coal mining machine drum 7 is the rear drum. Figure 4 As described above, the coal mining machine includes two coal mining machine drums 6 and 7, and the front drum is higher and the rear drum is lower during the coal mining process. If the coal mining machine drum 6 is higher and the coal mining machine drum 7 is lower during the coal mining process, the coal mining machine drum 6 is the front drum, and the coal mining machine drum 7 is the rear drum.

[0111] In the embodiment, the first target detection frame and the second target detection frame are obtained by a target detection algorithm. The first target detection frame and the second target detection frame are rectangular frames, each having four vertices. The corner points of the diagonal lines of the first target detection frame and the second target detection frame are the corresponding center points.

[0112] Step 304, determining that the direction from the rear drum to the front drum is the traction direction of the coal mining machine.

[0113] Optionally, the step 303 of comparing the coordinate of the first center point with the coordinate of the second center point to determine the front drum and the rear drum in the first drum and the second drum specifically includes:

[0114] Step 401, subtracting the longitudinal coordinate of the first center point from the longitudinal coordinate of the second center point to obtain a difference value;

[0115] Step 402a, if the difference value is greater than 0, determining that the first drum is the rear drum and the second drum is the front drum;

[0116] Step 402b, if the difference value is less than 0, determining that the first drum is the front drum and the second drum is the rear drum.

[0117] Figure 5 is a schematic diagram of a coal cutter drum target detection according to an exemplary embodiment. As shown, 8 is the first target detection frame, and 9 is the first center point. Figure 5

[0118] Optionally, the step 103 of determining the first support serial number of the front drum corresponding hydraulic support according to the position of the front drum corresponding target detection frame in the corresponding target detection image specifically comprises:

[0119] Step 501, obtaining the horizontal coordinate x of the center point of the target detection frame corresponding to the front drum c , and obtaining the pixel width D of the target detection image in the camera axis direction of the shooting device;

[0120] In this embodiment, the front drum of the coal mining machine is accurately positioned. The cutting interference phenomenon occurs in front of the coal mining machine, so in order to determine whether the cutting occurs, the front drum of the coal mining machine needs to be positioned according to the traction direction of the coal mining machine, and the traction direction of the coal mining machine has been described in the foregoing.

[0121] Considering that the first stage only realizes the preliminary positioning of the camera sequence (N) where the drum of the coal mining machine is located, in order to realize the close-range detection of the drum of the coal mining machine, it is necessary to further position the hydraulic support sequence where the front drum is located. The front drum of the coal mining machine is positioned and detected in close range to determine the first support serial number of the front drum corresponding hydraulic support.

[0122] Since the number of hydraulic supports in the detection image shot by each shooting device is fixed, the first support serial number of the front drum corresponding hydraulic support can be determined according to the area where the front drum of the coal mining machine is located in the detection image.

[0123] Step 502, determining at least one first coordinate interval according to the pixel width D and the preset number M of hydraulic supports, wherein each first coordinate interval contains a number of pixels

[0124] According to the preset number M of hydraulic supports corresponding to the shooting device, the pixel width D is divided into M intervals, and the number of pixels in each interval is In one possible embodiment, M is 3, which can be divided into three intervals and

[0125] Step 503, determining the first support serial number of the front drum corresponding hydraulic support according to the first coordinate interval where the horizontal coordinate x of the center point of the target detection frame corresponding to the front drum is located. c

[0126] ​​When the front roller corresponds to the first support serial number of the hydraulic support as 3N-2;

[0127] When the front roller corresponds to the first support serial number of the hydraulic support as 3N-1;

[0128] When the front roller corresponds to the first support serial number of the hydraulic support as 3N, wherein N is the serial number of the current shooting device.

[0129] Optionally, the step 105 of determining the second support serial number of the hydraulic support corresponding to the stretched guard plate according to the third target detection frame corresponding to the stretched guard plate in the guard plate detection image specifically includes:

[0130] Step 601, obtaining the serial number N of the target shooting device and the preset guard plate number A corresponding to the shooting device;

[0131] Since the number of guard plates in the detection image shot by each shooting device is fixed, the second support serial number of the hydraulic support corresponding to the guard plate can be determined according to the area where the guard plate is located in the detection image.

[0132] Step 602, obtaining the center point coordinates of the third target detection frame, and obtaining the pixel width D of the guard plate detection image in the camera axis direction of the shooting device;

[0133] Step 603, determining at least one second coordinate interval according to the pixel width D and the preset guard plate number A, wherein each second coordinate interval contains a number of pixels

[0134] According to the preset guard plate number A corresponding to the shooting device, the pixel width D is divided into A intervals, and the number of pixels in each interval is In a possible embodiment, A is 3, and the pixel width D can be divided into three intervals and

[0135] Step 604, determining the second support serial number of the hydraulic support corresponding to the stretched guard plate according to the second coordinate interval in which the abscissa of the center point coordinates of the third target detection frame is located.

[0136] When the second support serial number of the hydraulic support corresponding to the stretched guard plate is 3N-2;

[0137] When When the front drum corresponds to the first support sequence number 3N-1, the stretched state of the support plate corresponds to the second support sequence number 3N-1;

[0138] When the front drum corresponds to the first support sequence number 3N-1, the stretched state of the support plate corresponds to the second support sequence number 3N-1; When the front drum corresponds to the first support sequence number 3N-1, the stretched state of the support plate corresponds to the second support sequence number 3N-1.

[0139] Figure 6 is a kind of positioning map of hydraulic support of shooting equipment according to an exemplary embodiment, as Figure 6 As shown in the figure, 12 is a drum of coal mining machine; 13 is a shooting device; 14 is a hydraulic support. The visual angle of a single shooting device covers three hydraulic supports, and each hydraulic support corresponds to a support plate, that is, the visual angle of a single shooting device covers three support plates.

[0140] Optionally, the step 106 generates early warning information according to the first support sequence number and the second support sequence number, and the step specifically includes:

[0141] Step 701, the absolute value of the difference between the first support sequence number and the second support sequence number is taken as a distance parameter, and a warning distance interval is obtained;

[0142] Step 702, if the distance parameter is in the warning distance interval, a warning signal is generated.

[0143] Through the above steps, the first support sequence number corresponding to the front drum and the second support sequence number where the stretched state of the support plate is located are obtained.

[0144] Then, according to the distance information between the front drum and the stretched state of the support plate, a warning signal can be generated, one or more warning distance intervals are set, each warning distance interval corresponds to different warning levels, representing different degrees of danger. In one possible embodiment, the distance between one hydraulic support is set, the distance parameter is 1, a red warning signal is generated; the distance between two hydraulic supports is set, the distance parameter is 2, a yellow warning signal is generated; the distance between three hydraulic supports is set, the distance parameter is greater than or equal to 3, which is safe, and no warning signal is generated. The stretching of the support plate can also be adjusted in real time according to the distance information, which plays a role in the automatic production of fully mechanized working face.

[0145] Figure 8 is a kind of positioning warning device block diagram of coal mining working face equipment according to an exemplary embodiment. Refer to Figure 8 The device 800 includes:

[0146] The image acquisition module 810 is configured to capture a detection image, determine a target photographing device corresponding to the photographing device of the coal mining machine drum in the detection image, and determine a support serial number of a corresponding hydraulic support according to a serial number of the target photographing device, wherein the coal mining machine drum includes a front drum and a rear drum.

[0147] The traction direction determination module 820 is configured to acquire a first target detection frame of a first drum and a second target detection frame of a second drum in a target detection image of the target photographing device, and determine the front drum and the rear drum and a traction direction according to the first target detection frame and the second target detection frame in the first drum and the second drum.

[0148] The first support serial number determination module 830 is configured to determine a first support serial number of a hydraulic support corresponding to the front drum according to a position of a target detection frame corresponding to the front drum in a corresponding target detection image.

[0149] The guard plate detection module 840 is configured to determine a detection image captured by a photographing device located in the traction direction of the front drum as a guard plate detection image, and detect a guard plate in the guard plate detection image and a state of the guard plate.

[0150] The second support serial number determination module 850 is configured to determine a second support serial number of a hydraulic support corresponding to a guard plate in an extended state according to a third target detection frame corresponding to the guard plate in the extended state in the guard plate detection image.

[0151] The early warning module 860 is configured to generate early warning information according to the first support serial number and the second support serial number.

[0152] Figure 9 is a block diagram of an apparatus 900 according to an example embodiment. The apparatus 900 can be, for example, a mobile phone, a computer, a digital broadcast terminal, a message communication device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

[0153] Referring to Figure 9 , the apparatus 900 can include one or more of the following components: a processing component 902, a memory 904, a power supply component 906, a multimedia component 908, an audio component 910, an input / output (I / O) interface 912, a sensor component 914, and a communication component 916.

[0154] The processing component 902 generally controls the overall operations of the device 900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 902 can include one or more processors 920 to execute instructions delivered from the memory 904 to complete all or part of the steps of the methods described above. In addition, the processing component 902 can include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 can include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.

[0155] The memory 904 is configured to store various types of data to support the operations of the device 900. Examples of these data include instructions for any application or method operating on the device 900, contact data, phonebook data, messages, pictures, videos, and the like. The memory 904 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0156] The power component 906 provides power to the various components of the device 900. The power component 906 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 900.

[0157] The multimedia component 908 includes a screen providing an output interface between the device 900 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensors can not only sense a boundary of a touching or sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. The front and / or rear camera can receive external multimedia data when the device 900 is in an operating mode, such as a shooting mode or a video mode. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0158] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC) that is configured to receive an external audio signal when the device 900 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 904 or transmitted via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0159] The I / O interface 912 provides an interface between the processing component 902 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0160] The sensor component 914 includes one or more sensors for providing status assessments of various aspects of the device 900. For example, the sensor component 914 can detect an open / closed position of the device 900, relative positioning of components, such as a display and a keypad of the device 900, a change of position of the device 900 or a component of the device 900, presence or absence of user contact with the device 900, a change in orientation or acceleration / deceleration of the device 900, and a temperature change of the device 900, among other possibilities. The sensor component 914 can include a proximity sensor configured to detect presence of an object in proximity to the device 900 without any physical touch. The sensor component 914 can also include a light sensor (e.g., a CMOS or CCD image sensor) configured to work in an imaging application. In some embodiments, the sensor component 914 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0161] The communication component 916 is configured to facilitate wired or wireless communication between the device 900 and another device. The device 900 can access a wireless network based on a corresponding communication standard, such as WiFi, a cellular network (e.g., 2G, 3G, 4G, or 5G), or a combination thereof. In an example embodiment, the communication component 916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 916 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0162] In an exemplary embodiment, the apparatus 900 can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components, for performing the above-described methods.

[0163] In an exemplary embodiment, a storage medium including instructions, such as the memory 904 including instructions, is also provided, which can be executed by the processor 920 of the apparatus 900 to complete the above-described methods. Optionally, the storage medium can be a non-transitory computer-readable storage medium, such as a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0164] Figure 10 is a block diagram of an apparatus 1000 according to an exemplary embodiment. For example, the apparatus 1000 can be provided as a server. Referring to Figure 10 , the apparatus 1000 includes a processing component 1022, which further includes one or more processors, and a memory resource represented by the memory 1032, for storing instructions, such as an application program, executable by the processing component 1022. The application program stored in the memory 1032 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1022 is configured to execute the instructions to perform the above-described methods.

[0165] The apparatus 1000 can also include a power supply component 1026 configured to supply power to the apparatus 1000, a wired or wireless network interface 1050 configured to connect the apparatus 1000 to a network, and an input / output (I / O) interface 1058. The apparatus 1000 can operate based on an operating system stored in the memory 1032, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0166] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0167] It should be understood that the present disclosure is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the present disclosure. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method of positioning warning of a coal face equipment, characterized in that, The method comprises the following steps: Taking a detection image, determining a target shooting device corresponding to the detection image with a shearer drum, and determining the support serial number of a corresponding hydraulic support according to the serial number of the target shooting device, comprising: obtaining the serial number N of the target shooting device and the preset number M of hydraulic supports corresponding to the shooting device; determining the range of the support serial number of the hydraulic support corresponding to the serial number of the target shooting device as [(N-1)×M+1, N×M] according to the support serial number of the hydraulic support corresponding to the serial number N of the target shooting device and the preset number M of hydraulic supports, wherein the support serial number is a positive integer, and the shearer drum comprises a front drum and a rear drum; Obtaining a first target detection frame of a first drum and a second target detection frame of a second drum in a target detection image of the target shooting device, and determining the front drum and the rear drum and the traction direction of the shearer according to the first target detection frame and the second target detection frame in the first drum and the second drum; According to the position of the front roller corresponding target detection frame in the corresponding target detection image, the first support serial number of the front roller corresponding hydraulic support is determined, including: obtaining the horizontal coordinate x of the center point of the front roller corresponding target detection frame c , and obtaining the pixel width D of the target detection image in the shooting device machine axis direction; according to the pixel width D and the preset hydraulic support quantity M, at least one first coordinate interval is determined, wherein the pixel quantity contained in each first coordinate interval is According to the horizontal coordinate x of the center point of the front roller corresponding target detection frame c , the first support serial number of the front roller corresponding hydraulic support is determined. Determining a detection image of a guard plate as a guard plate detection image, detecting the guard plate in the guard plate detection image, and detecting the state of the guard plate; Determining the second support serial number of the hydraulic support corresponding to the stretched guard plate in the guard plate detection image according to the third target detection frame corresponding to the stretched guard plate; Generating a warning information according to the first support serial number and the second support serial number.

2. The method of claim 1, wherein, The step of driving the internal camera of the shooting device arranged on the hydraulic support to collect a detection image comprises: Initializing the pose of the internal camera in the shooting device, so that the internal camera faces the coal wall direction and shoots the detection image, wherein the shooting device is arranged on the hydraulic support.

3. The method of claim 1, wherein, The step of determining the target shooting device corresponding to the detection image with the shearer drum comprises: Setting the shearer drum detection flag of the shooting device corresponding to the detection image with the shearer drum to 1; Setting the shearer drum detection flag of the shooting device corresponding to the detection image without the shearer drum to 0.

4. The method of claim 1, wherein, The step of determining the front drum and the rear drum in the first drum and the second drum according to the first target detection frame and the second target detection frame, and determining the traction direction of the shearer comprises: Obtaining the first vertex coordinates of the vertices in the first target detection frame in the target detection image, and obtaining the second vertex coordinates of the vertices in the second target detection frame in the target detection image; Determining the coordinates of the first center point of the first target detection frame according to the first vertex coordinates, and determining the coordinates of the second center point of the second target detection frame according to the second vertex coordinates; Comparing the coordinates of the first center point and the coordinates of the second center point to determine the front drum and the rear drum in the first drum and the second drum; Determining the direction from the rear drum to the front drum as the traction direction of the shearer.

5. The method of claim 4, wherein, The step of comparing the coordinates of the first center point and the coordinates of the second center point to determine the front roller and the rear roller in the first roller and the second roller specifically comprises: Subtract the longitudinal coordinate of the first center point from the longitudinal coordinate of the second center point to obtain a difference value; If the difference value is greater than 0, it is determined that the first roller is the rear roller and the second roller is the front roller; If the difference value is less than 0, it is determined that the first roller is the front roller and the second roller is the rear roller.

6. The method of claim 1, wherein, The step of determining the second support serial number of the hydraulic support corresponding to the stretched state of the shield plate according to the third target detection frame corresponding to the stretched state of the shield plate in the shield plate detection image specifically comprises: Obtain the serial number N of the target shooting device and the preset number A of shield plates corresponding to the shooting device, wherein the state of the shield plate includes a folded state and a stretched state; Obtain the center point coordinates of the third target detection frame, and obtain the pixel width D of the shield plate detection image in the shooting device machine axis direction; At least one second coordinate interval is determined according to the pixel width D and the preset number of aprons A, wherein the number of pixels contained in each second coordinate interval is Determine the second support serial number of the hydraulic support corresponding to the stretched state of the shield plate according to the second coordinate interval in which the horizontal coordinate of the center point coordinates of the third target detection frame is located.

7. The method of claim 1, wherein, The step of generating a warning information according to the first support serial number and the second support serial number specifically comprises: Take the absolute value of the difference value between the first support serial number and the second support serial number as a distance parameter, and obtain a warning distance interval; If the distance parameter is in the warning distance interval, a warning signal is generated.

8. A positioning early warning device for a coal face apparatus, characterised in that, Comprise: An image acquisition module is used for shooting a detection image, determining a shooting device corresponding to the detection image with a shearer roller as a target shooting device, and determining a support serial number of a corresponding hydraulic support according to a serial number of the target shooting device, comprising: obtaining the serial number N of the target shooting device and the preset number M of hydraulic supports corresponding to the shooting device; determining the support serial number range of the hydraulic support corresponding to the serial number of the target shooting device as [(N-1)×M+1, N×M] according to the support serial number of the hydraulic support corresponding to the serial number N of the target shooting device and the preset number M of hydraulic supports, wherein the hydraulic support serial number is a positive integer, wherein the shearer roller comprises a front roller and a rear roller; A traction direction determination module is used for obtaining a first target detection frame of a first roller and a second target detection frame of a second roller in a target detection image of the target shooting device, and determining a front roller and a rear roller and a traction direction in the first roller and the second roller according to the first target detection frame and the second target detection frame; A first support sequence number determination module is configured to determine a first support sequence number of a hydraulic support corresponding to the front roller according to a position of a target detection frame corresponding to the front roller in a target detection image, and includes: obtaining an abscissa x of a center point of the target detection frame corresponding to the front roller c , and obtaining a pixel width D of the target detection image in a camera axis direction of the shooting device; determining at least one first coordinate interval according to the pixel width D and a preset number M of hydraulic supports, wherein a pixel number contained in each first coordinate interval is According to the abscissa x of the center point of the target detection frame corresponding to the front roller c , the first coordinate interval in which the center point is located is determined as the first support sequence number of the hydraulic support corresponding to the front roller. A shield plate detection module is used for determining a detection image of a shooting device located in the front roller traction direction as a shield plate detection image, detecting a shield plate in the shield plate detection image and a state of the shield plate; A second support serial number determination module is used for determining a second support serial number of a hydraulic support corresponding to a shield plate in a stretched state according to a third target detection frame corresponding to the shield plate in the stretched state in the shield plate detection image; An early warning module configured to generate early warning information based on the first stent serial number and the second stent serial number.

9. An electronic device, comprising: Comprise: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method of any one of claims 1-7.

10. A computer-readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enable the electronic device to perform the method of any one of claims 1-7.

11. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.

Images