Route control method and device, electronic equipment and storage medium

Abstract

The invention provides a route control method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining real-time images collected by a binocular camera, and the real-time images are images of two different visual angles of a top beam and an anchor cable of a same advanced hydraulic support; generating a depth map based on the real-time image; according to the depth map, the position information of the binocular camera and the geometric parameters of the current roadway, the distance value between the advanced hydraulic support top beam and the anchor cable is determined; and according to the distance value and a preset reference distance range, the walking route of the advanced hydraulic support is adjusted. Therefore, high-precision and real-time distance measurement can be achieved, the walking route of the advanced hydraulic support is adjusted in time, it is guaranteed that the top beam and the anchor cable have the best supporting effect, and safety and stability of a working face are guaranteed.

Description

Technical Field

[0001] This application relates to the technical fields of advanced hydraulic supports, image processing, etc., and particularly relates to a route control method, device, electronic device, and storage medium. Background Art

[0002] The roof beam and anchor cable of the advanced hydraulic support are important components of roadway support. The roof beam is used to directly support the roadway roof to prevent the roof from collapsing; while the anchor cable provides strong tensile support by anchoring into the deep and stable rock formation. The distance between the two has a direct impact on the support effect. If the distance is too large or too small, it is not conducive to the smooth progress of roadway tunneling and support construction. Therefore, it is necessary to timely adjust the walking route of the advanced hydraulic support to maintain the distance between the roof beam and the anchor cable. Summary of the Invention

[0003] This application aims to at least partly solve one of the technical problems in the related art.

[0004] To this end, the first object of this application is to propose a route control method to achieve high-precision and real-time distance measurement, timely adjust the walking route of the advanced hydraulic support, ensure the best support effect of the roof beam and the anchor cable, and guarantee the safety and stability of the working face.

[0005] The second object of this application is to propose a route control device.

[0006] The third object of this application is to propose an electronic device.

[0007] The fourth object of this application is to propose a computer-readable storage medium.

[0008] The fifth object of this application is to propose a computer program product.

[0009] To achieve the above object, the first aspect embodiment of this application proposes a route control method, including:

[0010] Obtain real-time images collected by a binocular camera, where the real-time images are images of two different perspectives including the roof beam and the anchor cable of the same advanced hydraulic support;

[0011] Generate a depth map based on the real-time images;

[0012] Determine the distance value between the roof beam of the advanced hydraulic support and the anchor cable according to the depth map, the position information of the binocular camera, and the geometric parameters of the current roadway;

[0013] Adjust the walking route of the advanced hydraulic support according to the distance value and the preset reference distance range.

[0014] To achieve the above object, an embodiment of the second aspect of the present application provides a route control device, including:

[0015] An acquisition module, configured to acquire real-time images collected by a binocular camera, where the real-time images are images of two different perspectives including the top beam and cable bolts of the same advanced hydraulic support;

[0016] A generation module, configured to generate a depth map based on the real-time images;

[0017] A determination module, configured to determine a distance value between the top beam of the advanced hydraulic support and the cable bolts according to the depth map, the position information of the binocular camera, and the geometric parameters of the current roadway;

[0018] A control module, configured to adjust the traveling route of the advanced hydraulic support according to the distance value and a preset reference distance range.

[0019] To achieve the above object, an embodiment of the third aspect of the present application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0020] The memory stores computer-executable instructions;

[0021] The processor executes the computer-executable instructions stored in the memory to implement the route control method provided in the embodiment of the first aspect of the present application

[0022] To achieve the above object, an embodiment of the fourth aspect of the present application provides a computer-readable storage medium, where computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by a processor, they are used to implement the route control method provided in the embodiment of the first aspect of the present application.

[0023] To achieve the above object, an embodiment of the fifth aspect of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the route control method provided in the embodiment of the first aspect of the present application.

[0024] The route control method, device, electronic device, and storage medium provided by the present application use the depth map generated by collecting images with a binocular camera to determine the distance between the top beam of the advanced hydraulic support and the cable bolts, realizing high-precision and real-time distance measurement. Moreover, based on the relationship between the measured distance and the safe distance range, the traveling route of the advanced hydraulic support is adjusted in a timely manner, so as to ensure that the top beam and cable bolts can maintain the best support effect, guarantee the safety and stability of the working face, and provide reliable technical support for the attitude adjustment of the advanced support.

[0025] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. Brief Description of the Drawings

[0026] The above-mentioned and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of embodiments in conjunction with the accompanying drawings, where:

[0027] Figure 1 is a schematic flowchart of a route control method provided by an embodiment of the present application;

[0028] Figure 2a is a top view of a roadway provided by an embodiment of the present application;

[0029] Figure 2b is a front view of a roadway provided by an embodiment of the present application;

[0030] Figure 3 is a schematic flowchart of another route control method provided by an embodiment of the present application;

[0031] Figure 4 is a schematic flowchart of another route control method provided by an embodiment of the present application; and

[0032] Figure 5 is a schematic structural diagram of a route control device provided by an embodiment of the present application. Detailed Description of the Embodiments

[0033] The embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application and should not be construed as limiting the present application.

[0034] The route control method and device of the embodiments of the present application will be described below with reference to the accompanying drawings.

[0035] In the prior art, when monitoring the running route of the advanced hydraulic support, sensors or lidar and other devices are installed to directly measure the distance between objects. The measurement cost is relatively high, and due to reasons such as environmental interference and measurement range limitation, the accuracy of the measurement results may be affected, and thus the walking route of the advanced hydraulic support cannot be accurately controlled, resulting in unstable roadway support effect.

[0036] The route control method proposed in this application can calculate the depth of the images collected by the binocular camera in real time, so as to monitor the distance between the top beam of the advanced hydraulic support and the cable anchor in real time, with relatively low cost. And by using a network based on deep learning algorithms to optimize the collected images, the route control method can maintain the accuracy of the distance measurement results in different environments, making the route control of the advanced hydraulic support more accurate and reliable.

[0037] Figure 1 It is a schematic flowchart of a route control method provided by an embodiment of this application.

[0038] As Figure 1 shown, the route control method includes but is not limited to the following steps:

[0039] Step 101, obtain the real-time images collected by the binocular camera.

[0040] Among them, the real-time images are images of two different perspectives containing the top beam and the cable anchor of the same advanced hydraulic support.

[0041] In the embodiment of this application, a binocular camera can be installed at the forefront of the advanced hydraulic support. During the walking process of the advanced hydraulic support, the top beam of the advanced hydraulic support and the nearest cable anchor are photographed synchronously, and two real-time images of different perspectives, left and right, are collected. The structure of the advanced hydraulic support and the position data of the cable anchor can be obtained from the real-time images.

[0042] It should be noted that in this application, a binocular camera is used to collect images. The binocular camera has the ability to collect images with high dynamic range (High-Dynamic Range, HDR), and can accurately capture the details of the top beam and the cable anchor in the case of uneven mine lighting. And the viewing distance of the binocular camera is 2 meters to 30 meters, and the field of view angle is 60°, which can meet the shooting requirements of the walking route of the advanced support in the gateway environment.

[0043] It should be noted that in this application, the binocular camera can collect images at regular intervals, or can also collect images every time the support needs to move forward, etc. This application does not make any limitations in this regard.

[0044] It should be noted that the advanced hydraulic support in this application can be a walking type advanced hydraulic support with a self-shifting function, which can walk forward in the roadway or working face to achieve continuous support. Or, in some possible embodiments, it can also be other types of advanced supports, which can ensure the support effect of the top beam and the cable anchor by adjusting the walking route. This application does not make any limitations in this regard.

[0045] The following describes the positional relationship between the advanced hydraulic support and the cable anchor in the working face in the roadway with reference to Figure 2.Figure 2a It is a plan view of the roadway, Figure 2b and it is a front view of the roadway.

[0046] In Figure 2a , the two longer solid lines on both sides represent the coal wall, the shorter solid lines inside the coal wall represent the roof beams of the advanced hydraulic supports, the dots inside the roof beams represent the cable bolts, and the middle dashed line is the gateway center line (i.e., the roadway center line). In the embodiments of the present application, it is necessary to measure the vertical distance between the cable bolt and the roof beam on the same side of the center line, which is the Figure 2a distance indicated by the double arrow in , so that the traveling route of the advanced hydraulic support can be controlled according to this distance.

[0047] In Figure 2b , the outermost rectangular frame represents the boundary of the roadway, the left and right boundaries are the coal walls, the lower boundary is the floor, and the upper boundary is the roof. The roof is a support structure for supporting rock strata or soil layers. The roof beam and the cable bolt are respectively used to support the roof and anchor into the deep and stable rock strata of the roof. Since the distance between the roof beam and the cable bolt will affect the force on the roof and thus the support effect, it is necessary to measure the distance between the roof beam and the cable bolt and make timely adjustments.

[0048] It should be noted that in Figure 2b , it can be seen that the roof beams of the advanced hydraulic supports on both sides of the center line have a cable bolt on each side of each roof beam. However, since the distance between the roof beams on both sides of the advanced hydraulic support is fixed and the width of each roof beam is also fixed, it is only necessary to measure the distance between one roof beam and one cable bolt on one side and adjust this distance to ensure that the distances between the roof beam and all other cable bolts are safe. Therefore, the binocular camera can be installed at the very front end in the traveling direction of the advanced hydraulic support to collect images including the nearest cable bolt and the roof beam.

[0049] Step 102, generate a depth map based on the real-time image.

[0050] It can be understood that since the binocular camera takes pictures of the same scene from different angles, there is a position difference, that is, parallax, between the projection points of the same object (such as the roof beam or the cable bolt) in the two real-time images. The parallax can be used to determine the distance of the object relative to the camera, that is, the depth.

[0051] Therefore, in the embodiments of the present application, the disparity between two real-time images can be calculated first, and then combined with the internal parameters, external parameters of the binocular camera and the baseline distance (i.e., the equal distance between the optical centers of the two cameras), the depth can be calculated using the principle of triangulation. That is, there is an inverse relationship between the disparity and the distance from the object to the camera (i.e., the depth). The larger the disparity, the closer the object, and the smaller the corresponding depth value; conversely, the smaller the disparity, the farther the object, and the larger the corresponding depth value, so that a depth map can be generated. Specifically, a stereo matching algorithm can be used to calculate the disparity, such as Block Matching (BM) or Semi-Global Matching (SGM), etc.

[0052] That is to say, the value of each pixel in the depth map represents the distance of the object corresponding to the pixel from the binocular camera in the three-dimensional space. The larger the value, the farther the object is from the camera, and the smaller the value, the closer the object is to the camera.

[0053] It should be noted that, in the embodiments of the present application, in order to improve the accuracy of the depth map, preprocessing such as denoising or brightness equalization can be performed on the real-time image before generating the depth map. Or, a model or network can also be used to optimize the generated depth map, refine the depth information in the depth map, and repair the edges and details of the depth map, etc.

[0054] Step 103, determine the distance value between the top beam of the advanced hydraulic support and the cable anchor according to the depth map, the position information of the binocular camera, and the geometric parameters of the current roadway.

[0055] Among them, the geometric parameters of the current roadway may include at least one of the roadway width, roadway height, roadway centerline based on the geodetic coordinates, etc. The roadway centerline can specify the driving direction of the roadway to ensure that the direction and position of the roadway meet the design requirements, and is set according to actual needs.

[0056] Among them, the position information of the binocular camera may include the coordinates of the binocular camera relative to the roadway centerline, and the height of the binocular camera from the floor, etc.

[0057] In the embodiments of the present application, after generating the depth map, the depth values corresponding to the roof beam and the cable bolt can be extracted from the depth map respectively. Then, based on the depth values, it can be determined whether the roof beam and the cable bolt are on the same horizontal plane. When the roof beam and the cable bolt are on the same horizontal plane and the viewing angle of the binocular camera is perpendicular to this plane, the actual distance between the roof beam and the cable bolt can be determined by calculating the pixel distance between the roof beam and the cable bolt in the depth map and combining the resolution of the depth map and the parameters of the camera. When the roof beam and the cable bolt are not on the same horizontal plane or the viewing angle of the binocular camera is not perpendicular to this plane, the coordinate transformation in the three-dimensional space can be utilized, combined with the position information of the binocular camera and the geometric parameters of the current roadway, to calculate the distance between the roof beam and the cable bolt. Thus, the distance measurement accuracy can be improved, effectively avoiding the errors that are prone to occur in complex environments during measurement. Especially during the process of the support following the machine, more accurate distance feedback can be provided.

[0058] It should be noted that in the embodiments of the present application, in order to improve the measurement accuracy and correct the errors in calculating the distance in the depth map, it is also possible to, each time the distance needs to be measured, adjust the shooting angles of different cameras to collect images multiple times to generate depth maps, so as to obtain multiple distance values. Then, the final distance value can be determined by taking the average of multiple distance values, or taking the value with the highest frequency among multiple distance values, etc., and then used for route control.

[0059] Step 104, adjust the traveling route of the advanced hydraulic support according to the distance value and the preset reference distance range.

[0060] Among them, the preset reference distance range refers to the range within which, when the distance between the roof beam and the cable bolt is within this range, a better support effect can be achieved, and it can be determined according to experience, the geometric constraint relationship between the roof beam and the cable bolt of the support, etc.

[0061] In the embodiments of the present application, it can be first determined whether the distance value determined based on the depth map is within the preset reference distance range. When the distance value is within the preset reference distance range, the advanced hydraulic support can continue to travel based on the current route. On the contrary, when the distance value is not within the preset reference distance range, at this time, it is necessary to adjust the distance between the roof beam and the cable bolt to ensure the support effect. Therefore, the traveling route of the advanced hydraulic support can be adjusted to adjust the distance between the roof beam and the cable bolt. For example, when the distance value is less than the minimum value in the reference distance range, it can be determined that the distance between the roof beam and the cable bolt is too small. Therefore, the traveling route of the advanced hydraulic support can be adjusted in the direction away from the cable bolt to increase the distance between the roof beam and the cable bolt, so that the adjusted distance is within the reference distance range.

[0062] In this embodiment, by using the depth map generated from the images collected by the binocular camera, the distance between the top beam of the advanced hydraulic support and the cable anchor is determined to achieve high-precision and real-time distance measurement. Moreover, based on the relationship between the measured distance and the safe distance range, the traveling route of the advanced hydraulic support is adjusted in a timely manner, so as to ensure that the top beam and the cable anchor can maintain the best support effect, guarantee the safety and stability of the working face, and provide reliable technical support for the attitude adjustment of the advanced support.

[0063] This embodiment provides another route control method. Figure 3 It is a schematic flowchart of another route control method provided by the embodiments of the present application.

[0064] As Figure 3 shown, this route control method may include but is not limited to the following steps:

[0065] Step 301, obtain the real-time image collected by the binocular camera.

[0066] For the detailed description of the above step 301, reference can be made to other embodiments of the present application, which will not be elaborated here.

[0067] It should be noted that in the embodiments of the present application, after obtaining the real-time image, the real-time image can be preprocessed first to improve the image quality and ensure the accuracy of subsequent depth calculation.

[0068] Optionally, the real-time image can be input into a denoising model to obtain the real-time image after noise removal.

[0069] In the embodiments of the present application, a denoising model trained by using a convolutional denoising network (CNN-Denoiser) can be adopted to remove the high-frequency noise in the image and obtain the real-time image after noise removal. The denoising model can effectively remove the noise interference in different environments through adaptive learning and retain the main structure information.

[0070] Alternatively, the brightness of the regions in the real-time image that are higher than the first threshold and / or lower than the second threshold can be adjusted according to the brightness of each region in the real-time image.

[0071] Among them, the first threshold refers to the region where the brightness is higher than this value is a strong light region. The second threshold refers to the region where the brightness is lower than this value is a low light region. Whether it is a strong light region or a low light region, the details in the image are relatively blurred, which affects the accuracy of subsequent depth calculation.

[0072] Therefore, in the embodiments of the present application, the brightness of the regions in the real-time image that are higher than the first threshold and / or lower than the second threshold can be adjusted through an adaptive brightness equalization algorithm, so as to improve the clarity of the image under different lighting conditions, make the brightness distribution of the entire image more uniform, and reduce the influence of external ambient light on the image quality.

[0073] It should be noted that, in some embodiments, the Sobel filter and the Canny edge detection algorithm can also be used to enhance the edge features of the support roof beam and the cable bolt in the acquired image. The above-provided real-time image preprocessing methods can be used simultaneously, which can more comprehensively improve the image quality.

[0074] Step 302, determine the disparity between real-time images.

[0075] In the embodiments of the present application, a suitable stereo matching algorithm, such as Block Matching (BM) or Semi-Global Matching (SGM), etc., can be selected to calculate the disparity between real-time images. Specifically, a pixel point can be selected in one of the real-time images, and then the pixel point most similar to this pixel point is traversed in the other real-time image. After that, the position difference between these two pixel points is calculated, so as to determine the corresponding disparity of these two pixel points. The above stereo matching process is performed on each pixel point in the real-time image to obtain a matrix containing the disparity values corresponding to all pixel points, that is, the disparity between real-time images.

[0076] Step 303, generate a depth map according to the disparity and the parameter information of the binocular camera.

[0077] Among them, the parameter information of the binocular camera can include the internal parameters of the binocular camera (such as focal length, optical center position, etc.) and external parameters (such as the relative position and angle between the cameras), as well as the known baseline distance (that is, the distance between the optical centers of the two cameras), etc.

[0078] In the embodiments of the present application, the principle of triangulation or existing depth calculation formulas, as well as the disparity and the parameter information of the binocular camera, can be used to calculate the depth value corresponding to each disparity value. After that, the depth value is mapped to the same resolution as the original real-time image to generate a depth map.

[0079] Step 304, input the depth map into a pre-trained optimization model to obtain an optimized depth map.

[0080] Among them, the optimization model is a deep learning network, which can be a combined model based on a Convolutional Neural Network (CNN) and a Generative Adversarial Network (GAN), used to optimize the details of the depth map and capable of improving the measurement accuracy in low-contrast or complex backgrounds.

[0081] In the embodiments of this application, after the depth map is input into the optimization model, the convolutional neural network can extract features from the depth map, focusing on extracting the geometric features of the front hydraulic support roof beam and the cable anchor in space. Through the design of multiple convolutional layers, the network can learn feature maps at different scales and effectively improve the spatial expression ability of features. The generative adversarial network can repair the depth details in the weak disparity regions of the depth map. The generative adversarial network can generate more accurate depth information in the edge and detail regions of the depth map through the adversarial training of the generator and the discriminator, which is beneficial to improving the distance measurement accuracy. Therefore, the optimization model can use a multi-layer self-attention mechanism to combine the global and local information of the image, improve the errors of the depth map caused by factors such as visual blur and insufficient light, output the optimized depth map, and thus is beneficial to improving the accuracy and reliability of the distance determined based on the depth map.

[0082] It should be noted that the training process of the optimization model can be a self-supervised learning process, through designing a synthetic dataset and using real data for unlabeled training. During the training process, the model can be optimized by calculating the reprojection error between the depth map and the actual scene.

[0083] It should be noted that in the embodiments of this application, the Pyramid Stereo Matching Network (PSMNet) algorithm can also be used to optimize the depth map. PSMNet adaptively adjusts the depth map under multi-scale features through a convolutional neural network, and can effectively handle the depth estimation errors in the depth map caused by disparity asymmetry, low-texture regions, and reflective surfaces. And PSMNet combines a pyramid structure with pixel-by-pixel matching to obtain accurate depth information, especially having a strong adaptability to the edge regions in complex scenes. Therefore, the depth map can be optimized by combining stereo matching and a deep learning network, and the geometric shape in the image can be adaptively adjusted to improve the depth estimation accuracy of the support roof beam and the cable anchor in the complex underground scene.

[0084] It should be noted that in order to adapt to the changes in external environmental factors such as the light intensity and dust concentration in the mine environment, in this application, the network parameters of the optimization model can also be adjusted and optimized in real time automatically through a reinforcement learning mechanism, so as to improve the optimization effect of the model on the depth map and ensure the stability and accuracy of the measurement process.

[0085] Optionally, in some embodiments of this application, the environmental data currently monitored by the sensor can be obtained first, and then the parameters of the optimization model can be updated based on the environmental data.

[0086] Among them, the environmental data refers to the change data of the environment around the advanced hydraulic support, and can include the detection values of the light intensity, dust concentration, etc.

[0087] In the embodiments of this application, the sensor can monitor the environmental data in real time, and when the distance needs to be measured each time, the currently monitored environmental data is transmitted to the optimization model in real time. Thus, after the sensor data and the image data are fused, they can be used as feedback to input into the optimization model to dynamically adjust the parameters of the model, so as to compensate for the influence of environmental changes on depth estimation. After receiving the environmental data, the optimization model can automatically adjust hyperparameters such as the learning rate and the number of convolutional layers in the depth map optimization process to enhance the adaptive ability of the network. Or it can also increase the weight of image enhancement for low light conditions to improve the depth estimation accuracy of low contrast regions.

[0088] Step 305: Determine the distance value between the top beam of the advanced hydraulic support and the cable anchor according to the optimized depth map, the position information of the binocular camera, and the geometric parameters of the current roadway.

[0089] Step 306: Adjust the traveling route of the advanced hydraulic support according to the distance value and the preset reference distance range.

[0090] For the detailed description of the above steps 304 and 305, reference can be made to other embodiments of this application, which will not be elaborated here.

[0091] In this embodiment, by calculating the disparity between the images collected in real time to generate a depth map and using a model to optimize the depth map, the accuracy and reliability of the depth map can be improved, providing reliable conditions for measuring the distance between the top beam and the cable anchor based on the depth map, and further improving the control accuracy of the traveling route of the advanced hydraulic support.

[0092] This embodiment provides another route control method. Figure 4 It is a schematic flowchart of another route control method provided by the embodiments of this application.

[0093] As Figure 4 shown, this route control method may include but is not limited to the following steps:

[0094] Step 401: Obtain the real-time image collected by the binocular camera.

[0095] Step 402: Generate a depth map based on the real-time image.

[0096] Step 403: Determine the distance value between the top beam of the advanced hydraulic support and the cable anchor according to the depth map, the position information of the binocular camera, and the geometric parameters of the current roadway.

[0097] For the detailed description of the above steps 401 to 403, reference can be made to other embodiments of this application, which will not be elaborated here.

[0098] Step 404: Generate an alarm message when the distance value exceeds the reference distance range.

[0099] Among them, the alarm message may include information such as the current distance value between the top beam and the cable anchor, the reference distance range, and the adjustment plan of the walking route.

[0100] In the embodiment of this application, the measurement result of the distance can be integrated with the mine equipment monitoring system to provide real-time feedback and alarm information. That is, when the distance value is less than the minimum value in the reference distance range or greater than the maximum value in the reference distance range, it is determined that the distance value exceeds the reference distance range. At this time, the support effect of the top beam and the cable anchor is poor, which may affect the production safety of the working face. Therefore, an alarm message needs to be generated to prompt the adjustment of the walking route of the advanced hydraulic support.

[0101] Step 405: Adjust the walking route of the advanced hydraulic support based on the alarm message.

[0102] In the embodiment of this application, the walking route of the advanced hydraulic support can be automatically adjusted according to the adjustment plan of the walking route in the alarm message to increase or decrease the distance between the top beam and the cable anchor. Or, the alarm message can be sent to the operation and maintenance personnel in the form of voice, text, or image, and then the operation and maintenance personnel send a route adjustment instruction, and then adjust the walking route of the advanced hydraulic support according to the instruction.

[0103] In this embodiment, by generating an alarm message when the distance value exceeds the reference distance range and then adjusting the walking route of the advanced hydraulic support based on the alarm message, the distance between the top beam and the cable anchor can be fed back in real time, which is convenient for timely adjusting the walking route of the advanced hydraulic support, thereby improving the efficiency of the operation and maintenance of the working face.

[0104] To implement the above embodiment, this application also proposes a route control device.

[0105] Figure 5 It is a schematic structural diagram of a route control device provided for the embodiment of this application.

[0106] As shown Figure 5 in the figure, the route control device includes:

[0107] An acquisition module 501, configured to acquire real-time images collected by a binocular camera, where the real-time images are images of two different perspectives including the roof beam and cable anchor of the same advanced hydraulic support;

[0108] A generation module 502, configured to generate a depth map based on the real-time images;

[0109] A determination module 503, configured to determine the distance value between the roof beam and the cable anchor of the advanced hydraulic support according to the depth map, the position information of the binocular camera, and the geometric parameters of the current roadway;

[0110] A control module 504, configured to adjust the traveling route of the advanced hydraulic support according to the distance value and a preset reference distance range.

[0111] Further, in a possible implementation manner of the embodiment of the present application, the generation module 502 may specifically be configured to:

[0112] Determine the parallax between the real-time images;

[0113] Generate a depth map according to the parallax and the parameter information of the binocular camera.

[0114] Further, in a possible implementation manner of the embodiment of the present application, the generation module 502 may further be configured to:

[0115] Input the depth map into a pre-trained optimization model to obtain an optimized depth map.

[0116] Further, in a possible implementation manner of the embodiment of the present application, the generation module 502 may further be configured to:

[0117] Acquire the environmental data currently monitored by the sensor;

[0118] Update the parameters of the optimization model based on the environmental data.

[0119] Further, in a possible implementation manner of the embodiment of the present application, the generation module 502 may further be configured to perform at least one of the following:

[0120] Input the real-time images into a denoising model to obtain real-time images with noise removed;

[0121] Adjust the brightness of the regions in the real-time images that are higher than the first threshold and / or lower than the second threshold according to the brightness of each region in the real-time images.

[0122] Further, in a possible implementation manner of the embodiment of the present application, the control module 504 may specifically be configured to:

[0123] Generate an alarm message when the distance value exceeds the reference distance range;

[0124] Adjust the traveling route of the advanced hydraulic support based on the alarm message.

[0125] It should be noted that the foregoing explanation of the embodiment of the route control method also applies to the route control device of this embodiment, and will not be elaborated here.

[0126] In the embodiment of the present application, by using the depth map generated by collecting images with a binocular camera to determine the distance between the roof beam of the advanced hydraulic support and the cable, high-precision and real-time distance measurement is realized. Moreover, based on the relationship between the measured distance and the safe distance range, the traveling route of the advanced hydraulic support is adjusted in a timely manner, so as to ensure that the roof beam and the cable can maintain the best support effect, guarantee the safety and stability of the working face, and provide reliable technical support for the attitude adjustment of the advanced support.

[0127] To implement the above embodiment, the present application also proposes an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided in the foregoing embodiment.

[0128] To implement the above embodiment, the present application also proposes a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the method provided in the foregoing embodiment.

[0129] To implement the above embodiment, the present application also proposes a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the method provided in the foregoing embodiment.

[0130] The collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved in the present application all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0131] It should be noted that personal information from users should be collected for legal and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after obtaining the informed consent of the user, including but not limited to notifying the user to read the user agreement / user notice and signing an agreement / authorization including authorizing relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others with access to the personal information data comply with their privacy policies and procedures.

[0132] This application is expected to provide an implementation scheme for users to selectively block the use or access of personal information data. That is, the present disclosure is expected to provide hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, the risk can be minimized by restricting data collection and deleting the data. In addition, when applicable, personal identifiers are removed from such personal information to protect the privacy of users.

[0133] In the description of the foregoing embodiments, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0134] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of this application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0135] Any process or method description shown in the flowchart or described in other ways herein can be understood to represent a module, segment, or part of code including one or more executable instructions for implementing a customized logic function or process, and the scope of the preferred implementation of this application includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in the reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of this application belong.

[0136] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definable sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in conjunction with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection parts with one or more wirings (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber devices, and portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other appropriate processing when necessary, and then storing it in a computer memory.

[0137] It should be understood that various parts of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0138] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0139] In addition, each functional unit in various embodiments of the present application may be integrated into a processing module, may exist physically alone for each unit, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0140] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A route control method, characterized in that: The following steps are involved: Acquire a real-time image collected by a binocular camera, wherein the real-time image is an image of two different perspectives including a top beam and an anchor cable of the same advanced hydraulic support; Based on the real-time image, generate a depth map; Determine the distance value between the top beam of the advance hydraulic support and the anchor cable according to the depth map, the position information of the binocular camera and the geometric parameters of the current lane; The travel route of the advanced hydraulic support is adjusted according to the distance value and a preset reference distance range.

2. The method according to claim 1, characterized in that The step of generating a depth map based on the real-time image comprises: determining a parallax between the real-time images; A depth map is generated according to the disparity and parameter information of the binocular camera.

3. The method according to claim 2, characterized in that After generating the depth map, the method further includes: The depth map is input into a pre-trained optimization model to obtain an optimized depth map.

4. The method according to claim 3, characterized in that Before inputting the depth map into the pre-trained optimization model, the method further includes: Get the environmental data currently monitored by the sensor; Based on the environmental data, parameters of the optimization model are updated.

5. The method according to any one of claims 1 to 4, characterized in that: Before generating a depth map based on the real-time image, the method further includes at least one of the following: Inputting the real-time image into a denoising model to obtain a real-time image after the noise is removed; According to the brightness of each area in the real-time image, the brightness of the area in the real-time image whose brightness is higher than a first threshold and / or lower than a second threshold is adjusted.

6. The method according to claim 1, characterized in that The step of adjusting the travel route of the advanced hydraulic support according to the distance value and the preset reference distance range includes: When the distance value exceeds the reference distance range, generating an alarm message; Based on the alarm information, the travel route of the advanced hydraulic support is adjusted.

7. A route control device, characterized in that: include: An acquisition module, used to acquire a real-time image collected by a binocular camera, wherein the real-time image is an image of two different perspectives including a top beam and an anchor cable of the same advanced hydraulic support; A generating module, used for generating a depth map based on the real-time image; A determination module, used to determine the distance value between the top beam of the advance hydraulic support and the anchor cable according to the depth map, the position information of the binocular camera and the geometric parameters of the current lane; The control module is used to adjust the travel route of the advanced hydraulic support according to the distance value and a preset reference distance range.

8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the route control method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the route control method according to any one of claims 1 to 6 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the route control method according to any one of claims 1 to 6.

Images