Vision-based automatic straightening control system and method for hydraulic support in fully mechanized mining working face

Abstract

The present application discloses a vision-based automatic straightening control system and method for hydraulic supports in a comprehensive mining working face, the system comprising: a cooperative target, a straightness detection camera, an edge computing terminal, a data transmission device, a hydraulic support electro-hydraulic control system and a digital push cylinder; wherein the cooperative target is installed on the front bridge of the base of each hydraulic support and is affixed with a reflective marker; the straightness detection camera is installed on the top of the hydraulic support of the comprehensive mining working face and its field of view is toward the direction of the hydraulic support; the edge computing terminal determines the target distance of the control; the hydraulic support electro-hydraulic control system controls each hydraulic support to move; the digital push cylinder monitors the movement and pressure information of the digital push cylinder. The technical solution provided by the present invention can realize the automatic detection and automatic adjustment control of the straightness of the hydraulic support, and does not require manual intervention, and has high straightening accuracy.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent coal mining, and in particular to a vision-based automatic straightening control system and method for hydraulic supports of a fully mechanized mining face. Background Art

[0002] Coal mining technology is an important technology in current coal production, and the intelligent and unmanned operation of the fully-mechanized mining face is the development direction of the current coal mining technology. With the continuous improvement of the intelligent level of the working face, the fully-mechanized mining face has realized automatic machine-following frame shifting and automatic coal cutting of the working face. However, in the coal mining process, the working face is required to have good straightness to ensure the quality of coal cutting by the coal mining machine. Therefore, the straightness detection and adaptive control system of the fully-mechanized mining face has become the focus and difficulty of the intelligent development of fully-mechanized mining. At present, the research on the automatic straightening technology of the working face is mostly focused on the posture feedback of the scraper conveyor. The invention patent CN111674838B proposes an automatic straightening device and method for the scraper conveyor body based on spatial position information capture, which can directly measure the relevant data of the straightness of the scraper conveyor body, and straighten the scraper conveyor body according to this data, thereby realizing the purpose of autonomous perception, autonomous analysis and rapid response of the scraper conveyor body straightening. Invention patent CN111441812A discloses an automatic straightening system and method for a coal mine fully mechanized mining face based on an inertial navigation system. The inertial navigation system is used to measure the straightness of the scraper conveyor, thereby calculating the correction amount of each hydraulic support and the actual moving distance of the hydraulic support. The moving control command is sent to the electro-hydraulic controller on each hydraulic support through the CAN network to control the moving, thereby completing the automatic straightening of the fully mechanized mining face. Invention patent CN105000328A uses an elastic rod installed between two hydraulic supports to measure the straightness of the hydraulic support after pulling the support, thereby realizing the automatic straightening of the scraper conveyor body. Invention patent CN112324483A proposes an automatic straightening system for hydraulic supports and a follower straightening system for scraper conveyors. The distance between the hydraulic support and the coal wall, and between the scraper conveyor and the hydraulic support is collected by an infrared distance sensor and transmitted to the controller. The movement and straightening of the scraper conveyor and the hydraulic support are realized through the control of the controller.

[0003] The research on the above-mentioned technology is mainly focused on the straightness detection of scraper conveyors or hydraulic supports. In the control of straightness, compensation control is mainly performed during the pushing and sliding process to ensure that the scraper conveyor is straight. The traditional straightness detection and control methods mainly have the following problems: (1) The traditional detection method mainly detects the straightness of the scraper conveyor. Although it reflects the straightness trend of the working surface, it lacks direct control variables for straightness control, and the control effect is difficult to guarantee. (2) The existing straightness control methods are all controlled during the scraper pushing process. Since the scraper is connected by multiple sections and is limited to each other, the adjustable range is limited. (3) The existing straightness control process lacks feedback. It only controls the movement stroke of the digital push cylinder through timing control. The overshoot is large, the control accuracy cannot be guaranteed, and manual intervention is required. Therefore, it is urgent to propose a control system and method that can accurately control the automatic straightening of the hydraulic support of the comprehensive mining working face. Summary of the invention

[0004] The present application provides a vision-based automatic straightening control system and method for hydraulic supports of a fully-mechanized mining working face, so as to at least solve the technical problem in the related art that automatic straightening of hydraulic supports of a fully-mechanized mining working face cannot be performed automatically and accurately.

[0005] The first embodiment of the present application proposes a vision-based automatic straightening control system for a hydraulic support in a fully mechanized mining face, the system comprising: a cooperative target, a straightness detection camera, an edge computing terminal, a data transmission device, a hydraulic support electro-hydraulic control system and a digital push cylinder;

[0006] The cooperative target is installed on the front bridge of the base of each hydraulic support and a reflective marker is posted on the cooperative target;

[0007] The straightness detection camera is installed on the top of the hydraulic support of the fully mechanized mining working face and the camera field of view is facing the direction of the hydraulic support, and is used to collect image information of the cooperative target on the hydraulic support and send the information to the edge computing terminal;

[0008] The edge computing terminal is used to receive the image information of the hydraulic support sent by the straightness detection camera, and determine the fitting straight line corresponding to each hydraulic support according to the received image information, and determine the target distance of the digital push cylinder control when the hydraulic support is pulled for the next cut based on the fitting straight line corresponding to each hydraulic support, and then send the target distance of the digital push cylinder control when the hydraulic support is pulled for the next cut to the data transmission device;

[0009] The data transmission device is used to receive the target distance of the digital push cylinder control when each hydraulic support is pulled and controlled, which is sent by the edge computing terminal, and send the received target distance of the digital push cylinder control when each hydraulic support is pulled and controlled, which is sent by the edge computing terminal, to the hydraulic support electro-hydraulic control system;

[0010] The hydraulic support electro-hydraulic control system is used to receive the target distance of the digital push cylinder control when each hydraulic support is pulled and controlled by the data transmission device, and then control each hydraulic support to move through the digital push cylinder based on the received target distance of the digital push cylinder control when each hydraulic support is pulled and controlled;

[0011] The digital push cylinder is used to detect the real-time movement information and pressure information of the digital push cylinder, and send the real-time movement information and pressure information to the hydraulic support electro-hydraulic control system to achieve closed-loop control.

[0012] The second embodiment of the present application proposes a vision-based automatic straightening control method for a hydraulic support in a fully mechanized mining working face, the method comprising:

[0013] Step S1: using a straightness detection camera to collect image information of a cooperative target on a hydraulic support, and sending the information to an edge computing terminal;

[0014] Step S2: The edge computing terminal calculates the rotation matrix and translation vector of each cooperative target in the camera coordinate system according to the cooperative target image information;

[0015] Step S3: Calculate the position and posture transformation relationship of each cooperative target in each collected image information, and obtain the transfer matrix from the cooperative target reference coordinate system to the image reference coordinate system in each image information;

[0016] Step S4: The edge computing terminals communicate with each other, obtain the image information set of each cooperative target in the comprehensive mining working face and its image reference transfer matrix, form a global association based on the overlapping cooperative targets in the image information, establish the optimal posture transfer path of the cooperative target node, and obtain the posture state of each cooperative target in the global coordinate system of the hydraulic support;

[0017] Step S5: performing straightness fitting of the hydraulic support on the working surface based on the posture state of each cooperative target in the global coordinate system of the hydraulic support, calculating the relative distance between the center point of the cooperative target and the fitted straight line, and calculating the target distance of the digital push cylinder control when the next hydraulic support performs the frame pulling control according to the digital push cylinder structure corresponding to the hydraulic support;

[0018] Step S6: The controlled target distance is transmitted to the hydraulic support electro-hydraulic control system, and then the hydraulic support electro-hydraulic control system controls the movement of each hydraulic support through the digital push cylinder based on the controlled target distance. At the same time, the hydraulic support electro-hydraulic control system collects the real-time movement information and pressure information of the digital push cylinder fed back by the digital push cylinder in real time.

[0019] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0020] The present application proposes a vision-based automatic straightening control system and method for hydraulic supports on a comprehensive mining working face, the system comprising: a cooperative target, a straightness detection camera, an edge computing terminal, a data transmission device, a hydraulic support electro-hydraulic control system and a digital push cylinder; the cooperative target is installed on the front bridge of the base of each hydraulic support and a reflective marker is posted on the cooperative target; the straightness detection camera is installed on the top of the hydraulic support on the comprehensive mining working face and the camera field of view is facing the direction of the hydraulic support, for collecting image information of the cooperative target on the hydraulic support and sending the information to the edge computing terminal; the edge computing terminal is used to receive the image information of the hydraulic support sent by the straightness detection camera, and determine the fitting straight line corresponding to each hydraulic support according to the received image information, and determine the target distance of the digital push cylinder control when each hydraulic support is pulled based on the fitting straight line corresponding to the hydraulic support for the next cut, and then Then, the target distance of the digital push cylinder control when the hydraulic support is pulled by the next knife is sent to the data transmission device; the data transmission device is used to receive the target distance of the digital push cylinder control when the hydraulic support is pulled by the edge computing terminal, and send the received target distance of the digital push cylinder control when the hydraulic support is pulled by the edge computing terminal to the hydraulic support electro-hydraulic control system; the hydraulic support electro-hydraulic control system is used to receive the target distance of the digital push cylinder control when the hydraulic support is pulled by the data transmission device, and then control each hydraulic support to move through the digital push cylinder based on the received target distance of the digital push cylinder control when the hydraulic support is pulled; the digital push cylinder is used to detect the real-time movement information and pressure information of the digital push cylinder, and send the real-time movement information and pressure information to the hydraulic support electro-hydraulic control system to realize closed-loop control. The technical solution provided by the present invention can realize the automatic detection and automatic adjustment control of the straightness of the hydraulic support, and does not require manual intervention, and has high straightening accuracy.

[0021] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A structural diagram of a vision-based automatic straightening control system for a hydraulic support in a fully mechanized mining face provided according to an embodiment of the present application;

[0024] Figure 2 A diagram showing the installation structure of a cooperative target and a straightness detection camera on a hydraulic support according to an embodiment of the present application;

[0025] Figure 3 A schematic diagram of a hydraulic support coordinate system established with the coal mining machine moving direction as the x-axis and the working face advancing direction as the y-axis according to an embodiment of the present application;

[0026] Figure 4 A structural diagram of a digital oil cylinder provided according to an embodiment of the present application;

[0027] Figure 5 A flowchart of a vision-based automatic straightening control method for a hydraulic support in a fully mechanized mining working face provided according to an embodiment of the present application;

[0028] Description of reference numerals:

[0029] The cooperative target 1, the straightness detection camera 2, the edge computing terminal 3, the data transmission device 4, the hydraulic support electro-hydraulic control system 5, the digital push cylinder 6, the cylinder body 6-1, the pressure sensor 6-2, the stroke sensor 6-3, the electromagnetic reversing valve 6-4, and the main controller 6-5, wherein the main controller 6-5 includes: a solution module 6-5-1, a power module 6-5-2, a communication module 6-5-3, a drive module 6-5-4, a hydraulic support 7, a base 7-1 of the hydraulic support, a top 7-2 of the hydraulic support, a target mounting bracket 8 and a mounting gimbal 9. DETAILED DESCRIPTION

[0030] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0031] In the vision-based automatic straightening control system and method of hydraulic supports on a comprehensive mining working face proposed in the present application, the system includes: a cooperative target, a straightness detection camera, an edge computing terminal, a data transmission device, a hydraulic support electro-hydraulic control system and a digital push cylinder; the cooperative target is installed on the front bridge of the base of each hydraulic support and a reflective marker is posted on the cooperative target; the straightness detection camera is installed on the top of the hydraulic support on the comprehensive mining working face and the camera field of view is facing the direction of the hydraulic support, which is used to collect image information of the cooperative target on the hydraulic support and send the information to the edge computing terminal; the edge computing terminal is used to receive the image information of the hydraulic support sent by the straightness detection camera, and determine the fitting straight line corresponding to each hydraulic support according to the received image information, and determine the target distance of the digital push cylinder control when each hydraulic support is pulled based on the fitting straight line corresponding to the hydraulic support for the next cut, and then Then, the target distance of the digital push cylinder control when the hydraulic support is pulled by the next knife is sent to the data transmission device; the data transmission device is used to receive the target distance of the digital push cylinder control when the hydraulic support is pulled by the edge computing terminal, and send the received target distance of the digital push cylinder control when the hydraulic support is pulled by the edge computing terminal to the hydraulic support electro-hydraulic control system; the hydraulic support electro-hydraulic control system is used to receive the target distance of the digital push cylinder control when the hydraulic support is pulled by the data transmission device, and then control each hydraulic support to move through the digital push cylinder based on the received target distance of the digital push cylinder control when the hydraulic support is pulled; the digital push cylinder is used to detect the real-time movement information and pressure information of the digital push cylinder, and send the real-time movement information and pressure information to the hydraulic support electro-hydraulic control system to realize closed-loop control. The technical solution provided by the present invention can realize the automatic detection and automatic adjustment control of the straightness of the hydraulic support, and does not require manual intervention, and has high straightening accuracy.

[0032] The following describes the vision-based automatic straightening control system and method of the hydraulic support of the comprehensive mining working face according to the embodiment of the present application with reference to the accompanying drawings.

[0033] Embodiment 1

[0034] Figure 1 The structure diagram of the automatic straightening control system of the hydraulic support of the fully mechanized mining working face based on vision is provided according to one embodiment of the present application, as shown in FIG. Figure 1 As shown, it may include: a cooperative target 1, a straightness detection camera 2, an edge computing terminal 3, a data transmission device 4, a hydraulic support electro-hydraulic control system 5 and a digital push cylinder 6;

[0035] The cooperative target 1 is installed on the front bridge of the base of each hydraulic support and a reflective marker is posted on the cooperative target 1;

[0036] The straightness detection camera 2 is installed on the top of the hydraulic support of the fully mechanized mining working face and the camera field of view is facing the direction of the hydraulic support, and is used to collect image information of the cooperative target 1 on the hydraulic support and send the information to the edge computing terminal 3;

[0037] The edge computing terminal 3 is used to receive the image information of the hydraulic support sent by the straightness detection camera 2, and determine the fitting straight line corresponding to each hydraulic support according to the received image information, and determine the target distance of the digital push cylinder control when the hydraulic support is pulled for the next cut based on the fitting straight line corresponding to each hydraulic support, and then send the target distance of the digital push cylinder control when the hydraulic support is pulled for the next cut to the data transmission device 4;

[0038] The data transmission device 4 is used to receive the target distance of the digital push cylinder control during the hydraulic support rack pulling control sent by the edge computing terminal 3, and send the received target distance of the digital push cylinder control during the hydraulic support rack pulling control sent by the edge computing terminal 3 to the hydraulic support electro-hydraulic control system 5;

[0039] The hydraulic support electro-hydraulic control system 5 is used to receive the target distance of the digital push cylinder control when each hydraulic support is pulled and controlled by the data transmission device 4, and then control each hydraulic support to move through the digital push cylinder based on the received target distance of the digital push cylinder control when each hydraulic support is pulled and controlled;

[0040] The digital push cylinder 6 is used to detect the real-time movement information and pressure information of the digital push cylinder, and send the real-time movement information and pressure information to the hydraulic support electro-hydraulic control system 5 to achieve closed-loop control.

[0041] It should be pointed out that Figure 1 The system structure diagram shown is a system structure diagram of only one cooperative target 1, a straightness detection camera 2, an edge computing terminal 3, a data transmission device 4, a hydraulic support electro-hydraulic control system 5 and a digital push cylinder 6. Figure 1 This is only for illustration and is not intended to limit the embodiments of the present application.

[0042] In the embodiments of the present disclosure, Figure 2The figure shows the installation structure diagram of the cooperative target 1 and the straightness detection camera 2 on the hydraulic support 7. The cooperative target 1 can be installed on the front bridge of the base 7-1 of each hydraulic support through the target mounting bracket 8, and the straightness detection camera 2 can be installed on the top 7-2 of the hydraulic support through the installation pan / tilt 9 and the camera field of view is facing the direction of the hydraulic support 7.

[0043] In the disclosed embodiment, the field of view of the installed straightness detection camera 2 is such that it can observe the cooperative targets 1 on n hydraulic supports and the number of the same cooperative targets that can be photographed by adjacent straightness detection cameras 2 is greater than or equal to 2; wherein n is greater than or equal to 2.

[0044] In the disclosed embodiment, the cooperative target 1 adopts an ArUco target, which is provided with a center point and three focusing points.

[0045] The cooperative target 1 can be installed on the front bridge of the base of the hydraulic support through a target installation bracket.

[0046] It should be noted that the number of the edge computing terminals 3 is set based on the length of the comprehensive mining work and data interaction and aggregation can be performed between the edge computing terminals 3 .

[0047] Among them, the edge computing terminal 3 can be installed on the top beam of the hydraulic support of the comprehensive mining working face by installing a pan-tilt platform.

[0048] In the disclosed embodiment, determining the fitting straight line corresponding to each hydraulic support according to the received image information includes:

[0049] The edge computing terminal 3 calculates the rotation matrix and translation vector of each cooperative target 1 in the camera coordinate system according to the received image information of each cooperative target;

[0050] Calculate the position and posture transformation relationship of each target in each of the image information, and obtain the transfer matrix from the cooperative target reference coordinate system to the image reference coordinate system in each of the image information;

[0051] Each edge computing terminal 3 communicates with each other, obtains the image information set of each cooperative target in the fully mechanized working face and the image reference transfer matrix of each cooperative target 1, forms a global association based on the overlapping cooperative targets 1 in the image information, establishes the optimal posture transfer path of the cooperative target node, and obtains the posture state of each cooperative target 1 in the global coordinate system of the hydraulic support;

[0052] Based on the position and posture information of each cooperative target 1 in the global coordinate system, the fitting straight line corresponding to each hydraulic support of the fully mechanized mining working face is determined.

[0053] Among them, the global coordinate system of the hydraulic support is established with the position of the center point of the cooperative target on the first hydraulic support in the comprehensive mining working face as the origin, the moving direction of the coal mining machine as the x-axis, the advancing direction of the working face as the y-axis, and the height direction of the support as the z-axis.

[0054] Specifically, the method of determining the fitting straight line corresponding to each hydraulic support of the fully mechanized mining face based on the posture information of the cooperative target 1 in the global coordinate system includes:

[0055] Determine the center point function of each hydraulic support cooperation target in the fully mechanized mining face under the global coordinate system;

[0056] It should be noted that the center point function of the support frame cooperation target is obtained by connecting the center points of the cooperation targets 1 through line segments.

[0057] For example, Figure 3 As shown, the center point of the cooperative target 1 on the first hydraulic support in the fully mechanized working face is taken as the origin, the moving direction of the coal mining machine is the x-axis, and the advancing direction of the working face is the y-axis. The coordinate system of the hydraulic support is established. The coordinates corresponding to the first hydraulic support are (x1, y1). The center points of the cooperative targets 1 are connected by line segments, and the center point function y=f(x) of the hydraulic support target 1 can be obtained. Then the horizontal coordinate of the digital displacement cylinder corresponding to the i-th hydraulic support is x i , then the coordinates of the center point of the corresponding cooperative target 1 are (x i ,f(x i )).

[0058] Based on the center point function, the fitting straight line corresponding to each hydraulic support of the fully mechanized mining working face is determined.

[0059] It should be noted that the coordinates of the center points of each cooperative target are obtained through the center point function, and the fitting straight lines corresponding to each hydraulic support of the comprehensive mining working face are determined based on the coordinates.

[0060] In the embodiment of the present disclosure, the method of determining the target distance of the digital push cylinder control when the hydraulic supports are pulled up for the next cut based on the fitting straight line corresponding to the hydraulic supports includes:

[0061] The center point function is used to calculate the relative distance between the center point of the cooperative target and the fitted straight line.

[0062] Based on the relative distance and the structure of the digital push cylinder corresponding to the hydraulic support, the target distance of the digital push cylinder control when each hydraulic support is pulled to control the next cut is calculated.

[0063] It should be noted that, because the position of the digital push cylinder is not horizontal, the distance the hydraulic support moves forward is not completely consistent with the distance the digital push cylinder pushes out. Therefore, the target distance L' controlled by the digital push cylinder when the qth hydraulic support is pulled is calculated according to the following formula: Among them, L is the length of the digital push cylinder at the current moment, D is the relative distance from the center point of the cooperative target corresponding to the qth hydraulic support to the fitted straight line, h is the height between the hinge point of the digital push cylinder and the ground.

[0064] It should be noted that if Figure 4 As shown, the digital push cylinder 6 is composed of a cylinder body 6-1, a pressure sensor 6-2, a stroke sensor 6-3, an electromagnetic reversing valve 6-4, and a main controller 6-5, wherein the main controller 6-5 includes: a solution module 6-5-1, a power supply module 6-5-2, a communication module 6-5-3 and a drive module 6-5-4.

[0065] The pressure sensor 6-2 and stroke sensor 6-3 are used to monitor the real-time movement information and pressure information of the digital push cylinder. The main controller 6-5 controls the start and stop time of the driver 6-5-4 according to the value of the single push stroke received by the communication module 6-5-3, thereby controlling the start and stop time of the electromagnetic reversing valve 6-4 to achieve precise control of the position of the cylinder body 6-1.

[0066] To sum up, the automatic straightening control system of the hydraulic support of the comprehensive mining working face proposed in the present invention uses a digital push cylinder, which avoids the problem that during the movement of the digital push cylinder, due to the slow response of the electro-hydraulic reversing valve, there is a delay in the electro-hydraulic control system sending the reversing command to the reversing valve action, resulting in the actual pushing distance of the digital push cylinder being larger than the value set by the automatic straightening system, resulting in the digital push cylinder being unable to accurately run to the specified position after sending the target position command, and cumulative errors occurring after multiple pushes, requiring manual adjustment. As a result, the straightness of the hydraulic support is automatically detected and adjusted in the pulling frame without the need for manual intervention, and the straightening accuracy is high.

[0067] Embodiment 2

[0068] Figure 5 According to an embodiment of the present application, a vision-based automatic straightening control method for a hydraulic support in a fully mechanized mining working face is provided, such as Figure 5 As shown, it may include:

[0069] Step S1: using a straightness detection camera to collect image information of a cooperative target on a hydraulic support, and sending the information to an edge computing terminal;

[0070] Step S2: The edge computing terminal calculates the rotation matrix and translation vector of each cooperative target in the camera coordinate system according to the cooperative target image information;

[0071] Step S3: Calculate the position and posture transformation relationship of each cooperative target in each collected image information, and obtain the transfer matrix from the cooperative target reference coordinate system to the image reference coordinate system in each image information;

[0072] Step S4: The edge computing terminals communicate with each other, obtain the image information set of each cooperative target in the comprehensive mining working face and its image reference transfer matrix, form a global association based on the overlapping cooperative targets in the image information, establish the optimal posture transfer path of the cooperative target node, and obtain the posture state of each cooperative target in the global coordinate system of the hydraulic support;

[0073] Step S5: performing straightness fitting of the hydraulic support on the working surface based on the posture state of each cooperative target in the global coordinate system of the hydraulic support, calculating the relative distance between the center point of the cooperative target and the fitting straight line, and calculating the target distance of the digital push cylinder control when the next hydraulic support performs the frame pulling control according to the digital push cylinder structure corresponding to the hydraulic support;

[0074] Step S6: The controlled target distance is transmitted to the hydraulic support electro-hydraulic control system, and then the hydraulic support electro-hydraulic control system controls the movement of each hydraulic support through the digital push cylinder based on the controlled target distance. At the same time, the hydraulic support electro-hydraulic control system collects the real-time movement information and pressure information of the digital push cylinder fed back by the digital push cylinder in real time.

[0075] It should be noted that the collected image information has overlapping cooperative targets.

[0076] In summary, the vision-based automatic straightening control method of the hydraulic support in the comprehensive mining working face proposed in the present invention enables automatic detection and automatic adjustment of the straightness of the hydraulic support in the pulling frame, without the need for manual intervention, and with high straightening accuracy.

[0077] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0078] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0079] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A vision-based automatic straightening control system for hydraulic supports in fully mechanized mining working faces, characterized in that: The system includes: a cooperative target, a straightness detection camera, an edge computing terminal, a data transmission device, an electro-hydraulic control system of a hydraulic support and a digital push cylinder; The cooperative target is installed on the front bridge of the base of each hydraulic support and a reflective marker is posted on the cooperative target; The straightness detection camera is installed on the top of the hydraulic support and the camera field of view is facing the direction of the hydraulic support, and is used to collect image information of the cooperative target on the hydraulic support and send the information to the edge computing terminal; The edge computing terminal is used to receive the image information of the hydraulic support sent by the straightness detection camera, and determine the fitting straight line corresponding to each hydraulic support according to the received image information, and determine the target distance of the digital push cylinder control when the hydraulic support is pulled for the next cut based on the fitting straight line corresponding to each hydraulic support, and then send the target distance of the digital push cylinder control when the hydraulic support is pulled for the next cut to the data transmission device; The data transmission device is used to receive the target distance of the digital push cylinder control when each hydraulic support is pulled and controlled, which is sent by the edge computing terminal, and send the received target distance of the digital push cylinder control when each hydraulic support is pulled and controlled, which is sent by the edge computing terminal, to the hydraulic support electro-hydraulic control system; The hydraulic support electro-hydraulic control system is used to receive the target distance of the digital push cylinder control when each hydraulic support is pulled and controlled by the data transmission device, and then control each hydraulic support to move through the digital push cylinder based on the received target distance of the digital push cylinder control when each hydraulic support is pulled and controlled; The digital push cylinder is used to detect the real-time movement information and pressure information of the digital push cylinder, and send the real-time movement information and pressure information to the hydraulic support electro-hydraulic control system to achieve closed-loop control.

2. The system according to claim 1, characterized in that The field of view of the installed straightness detection camera is such that the cooperative targets on n hydraulic supports can be observed and the number of the same cooperative targets that can be photographed by adjacent straightness detection cameras is greater than or equal to 2; Wherein, n is greater than or equal to 2.

3. The system according to claim 1, characterized in that The cooperative target adopts an ArUco target, which is provided with a center point and three focusing points.

4. The system according to claim 1, characterized in that The number of the edge computing terminals is set based on the length of the comprehensive mining work and data interaction and aggregation can be performed between the edge computing terminals.

5. The system according to claim 4, characterized in that The step of determining the fitting straight line corresponding to each hydraulic support according to the received image information comprises: The edge computing terminal calculates the rotation matrix and translation vector of each cooperative target in the camera coordinate system based on the received image information of each cooperative target; Calculate the position and posture transformation relationship of each target in each of the image information, and obtain the transfer matrix from the cooperative target reference coordinate system to the image reference coordinate system in each of the image information; Each edge computing terminal communicates with each other, obtains the image information set of each cooperative target in the comprehensive mining working face and its image reference transfer matrix, forms a global association based on the overlapping cooperative targets in the image information, establishes the optimal posture transfer path of the cooperative target node, and obtains the posture state of each cooperative target in the global coordinate system of the hydraulic support; Based on the position information of each cooperative target in the global coordinate system, the fitting straight line corresponding to each hydraulic support in the fully mechanized mining working face is determined.

6. The system according to claim 5, characterized in that The global coordinate system of the hydraulic support is established with the position of the center point of the cooperative target on the first hydraulic support in the comprehensive mining working face as the origin, the coal mining machine moving direction as the x-axis, the working face advancing direction as the y-axis, and the support height direction as the z-axis.

7. The system according to claim 6, characterized in that The method of determining the fitting straight line corresponding to each hydraulic support of the fully mechanized mining working face based on the posture information of the cooperative target in the global coordinate system includes: Determine the center point function of each hydraulic support cooperation target in the fully mechanized mining face under the global coordinate system; Based on the center point function, the fitting straight line corresponding to each hydraulic support of the fully mechanized mining working face is determined.

8. The system according to claim 7, characterized in that The method of determining the target distance of the digital push cylinder control when the hydraulic supports are pulled and controlled for the next cut based on the fitting straight line corresponding to the hydraulic supports includes: The center point function is used to calculate the relative distance between the center point of the cooperative target and the fitted straight line. Based on the relative distance and the structure of the digital push cylinder corresponding to the hydraulic support, the target distance of the digital push cylinder control when each hydraulic support is pulled to control the next cut is calculated.

9. A vision-based automatic straightening control method for hydraulic supports in fully mechanized mining working faces, characterized in that: The method comprises: Step S1: using a straightness detection camera to collect image information of a cooperative target on a hydraulic support, and sending the information to an edge computing terminal; Step S2: The edge computing terminal calculates the rotation matrix and translation vector of each cooperative target in the camera coordinate system according to the cooperative target image information; Step S3: Calculate the position and posture transformation relationship of each cooperative target in each collected image information, and obtain the transfer matrix from the cooperative target reference coordinate system to the image reference coordinate system in each image information; Step S4: The edge computing terminals communicate with each other, obtain the image information set of each cooperative target in the comprehensive mining working face and its image reference transfer matrix, form a global association based on the overlapping cooperative targets in the image information, establish the optimal posture transfer path of the cooperative target node, and obtain the posture state of each cooperative target in the global coordinate system of the hydraulic support; Step S5: performing straightness fitting of the hydraulic support on the working surface based on the posture state of each cooperative target in the global coordinate system of the hydraulic support, calculating the relative distance between the center point of the cooperative target and the fitting straight line, and calculating the target distance of the digital push cylinder control when the next hydraulic support performs the frame pulling control according to the digital push cylinder structure corresponding to the hydraulic support; Step S6: The controlled target distance is transmitted to the hydraulic support electro-hydraulic control system, and then the hydraulic support electro-hydraulic control system controls the movement of each hydraulic support through the digital push cylinder based on the controlled target distance. At the same time, the hydraulic support electro-hydraulic control system collects the real-time movement information and pressure information of the digital push cylinder fed back by the digital push cylinder in real time.

10. The straightening control method according to claim 9, characterized in that: The collected image information has overlapping cooperative targets.

Images