Working face control system, method, apparatus, storage medium and computer equipment

Abstract

This invention discloses a working face control system, method, apparatus, storage medium, and computer equipment. The system includes a control unit group consisting of multiple control units, each mounted on a different functional component and connected via a bus. The control unit group includes a target detection model, and each control unit has a different sub-model. Each sub-model executes one of a pre-defined sub-step based on the target detection steps. The target detection model executes the target detection steps and obtains the working information of the coal mining machine. The sub-models control the movement state of their corresponding functional components based on the working information of the coal mining machine. The control units also include a wireless communication module, which enables wireless communication based on a preset heterogeneous networking technology in case of communication anomalies between two interoperating control units. This method can reduce the hardware cost of coal mining machine control in fully mechanized mining faces.

Description

Technical Field

[0001] This invention relates to the field of automatic control technology, and in particular to a working face control system, method, apparatus, storage medium, and computer equipment. Background Technology

[0002] With the development of automatic control technology, control systems have emerged to enable autonomous and intelligent decision-making for coal mining equipment in fully mechanized mining faces. These systems use various sensors installed within the mining face to collect data on the working status of the coal mining machine and the working conditions of the face, controlling the movement of functional components such as the hydraulic supports. The computational unit used for autonomous and intelligent decision-making is often located within the control unit on the fully mechanized mining face that controls each functional component, such as the control unit that controls the hydraulic supports' pushing and moving operations.

[0003] However, in fully mechanized mining operations, the control systems for functional components rely heavily on multiple sensors, high-bandwidth networks, and multi-level subsystems to support intelligent operation of the longwall face in order to achieve autonomous and intelligent decision-making. However, when the longwall mining equipment is located in thin or extremely thin coal seams, the hydraulic support control system is limited by the space available in the hydraulic support and cannot deploy multiple sensors.

[0004] In this context, the image recognition module is crucial for enabling autonomous and intelligent decision-making for various functional components of the fully mechanized mining face. However, the module capable of image recognition places extremely high demands on the computing power of the control system, resulting in very high hardware costs for achieving autonomous and intelligent decision-making in the fully mechanized mining face. Summary of the Invention

[0005] In view of this, this application provides a working face control system, method, apparatus, storage medium and computer equipment, the main purpose of which is to solve the technical problem of excessively high hardware costs for achieving autonomous intelligent decision-making in fully mechanized mining faces.

[0006] According to a first aspect of the present invention, a working face control system is provided, comprising a control unit group consisting of multiple control units, wherein the multiple control units are respectively disposed on multiple functional components on a fully mechanized mining working face, and the multiple control units are connected based on a bus.

[0007] The control unit group is equipped with a target detection model, and each control unit is equipped with a different target detection sub-model. The target detection sub-model is used to execute one of the multiple sub-steps pre-divided based on the target detection steps.

[0008] The target detection model is used to execute the target detection steps and obtain the working information of the coal mining machine. The target detection sub-model is used to control the motion state of the corresponding functional components based on the working information of the coal mining machine.

[0009] The control unit is also equipped with a wireless communication module, which is used to realize wireless communication based on a preset heterogeneous networking technology when the communication status of the two control units communicating with each other is abnormal.

[0010] According to a second aspect of the present invention, a working face control method is provided, applied to the system performing the above, comprising:

[0011] The image information of multiple structural components on the coal mining machine is acquired in real time, and the image information is input into a pre-trained target detection model to obtain the position information of each structural component;

[0012] Determine the position of the coal mining machine in the coal mining face and the working stage of the coal mining machine;

[0013] The location of the coal mining machine, the working stage, the location information of each structural component, and the sensor information collected by at least one sensor set in the fully mechanized mining face are respectively input into the pre-trained component motion prediction model set in each control unit to obtain the motion control data of each functional component.

[0014] Based on the motion control data, the operating state of each of the functional components is controlled.

[0015] According to a third aspect of the present invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described working surface control method.

[0016] According to a fourth aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described working surface control method.

[0017] This invention provides a working face control system, method, apparatus, storage medium, and computer equipment. It can pre-decompose the target detection model into multiple sub-models, which are distributed among the control units of multiple functional components in the fully mechanized mining face. These control units can interact with each other via a bus. Furthermore, the working information of the coal mining machine is obtained through the target detection step, and the operating status of each functional component is controlled based on this information. Simultaneously, the control units also have the ability to communicate based on heterogeneous networking, improving the system's resilience to abnormal situations such as bus interruptions. This application distributes the computing power required for target detection across multiple control units in the functional components, significantly reducing the computing power requirements of individual control units and thus lowering the hardware cost for achieving autonomous intelligent decision-making in the fully mechanized mining face.

[0018] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0020] Figure 1 This diagram illustrates the structure of a working face control system provided in an embodiment of the present invention.

[0021] Figure 2 A flowchart illustrating a working face control method provided by an embodiment of the present invention is shown;

[0022] Figure 3 This figure shows a schematic diagram of the structure of a working face control device provided in an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of another working face control device provided in an embodiment of the present invention is shown.

[0024] The markings in the image are as follows:

[0025] 110. Bus; 120. Control unit group; 121. Control unit. Detailed Implementation

[0026] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0027] Currently, autonomous intelligent decision-making technology is a crucial development direction for coal mining machines used in fully mechanized mining operations. This technology uses various sensors installed within the coal face to collect data on the machine's operating status and the working conditions of the face, controlling the movement of functional components such as hydraulic supports. The computing units used to achieve autonomous intelligent decision-making are often control units on the coal mining machine that control each functional component, such as the hydraulic support control unit. However, in fully mechanized mining operations, the control systems for functional components rely heavily on hardware-based multiple sensors, high-bandwidth networks, and multi-level subsystems to support intelligent operation of the longwall face. However, when the longwall mining equipment is located in thin or extremely thin coal seams, the hydraulic support control system is limited by the space available in the hydraulic supports, preventing the deployment of multiple sensors. In this situation, image recognition modules are crucial for achieving autonomous intelligent decision-making for the various functional components of the coal mining equipment. However, modules capable of image recognition require extremely high computational power from the control system, resulting in very high hardware costs for achieving autonomous intelligent decision-making in coal mining equipment.

[0028] To address the above problems, in one embodiment, such as Figure 1 As shown, a working face control system is provided. The system includes a control unit group 120 consisting of multiple control units 121. The multiple control units 121 are respectively disposed on multiple functional components (not shown in the figure) on the fully mechanized mining working face, and the multiple control units 121 are connected via a bus 110.

[0029] The functional components may include equipment such as hydraulic supports, coal mining machines, and scraper conveyors in the fully mechanized mining face. Furthermore, the control unit 121 can be a control device with certain computing capabilities that controls the movement of the functional components. Further, the control unit group 120 is equipped with a target detection model, and each control unit 121 is equipped with different target detection sub-models derived from the target detection model. The target detection sub-model is used to execute one of the multiple sub-steps pre-divided based on the target detection steps. The target detection model can be a pre-trained convolutional neural network model, capable of determining the position of each structural component in the image based on the input image information. Further, each target detection sub-model is set in a pre-selected control unit 121, and all control units 121 in the control unit group 120 are equipped with all target detection sub-models of the target detection model. The target detection model can be a YOLOv5 model, whose network structure can be divided into an Input network, a Backbone network, a Neck network, and a Head network. In practical applications, the Input network, Backbone network, Neck network, and Head network of the YOLOv5 model can be used as sub-steps of the target detection step. Each sub-step is set in a target detection step sub-model, and the target detection step sub-models are set in different control units 121 within the control unit group 120. The control units 121 where each sub-model is located can communicate with each other through the data bus 110.

[0030] Furthermore, the target detection model is used to perform target detection steps and obtain the working information of the coal mining machine. This working information includes the positional information of each structural component of the coal mining machine, as well as the machine's current position and working stage. The structural components of the coal mining machine can include the left rocker arm, left coal drum, right rocker arm, and right coal drum. Specifically, multiple image acquisition devices installed on the coal mining machine can be used to acquire image information of these structural components. The focal plane of the image acquisition device can be parallel to the motion plane of the structural component to fully capture its motion state and position. Further, multiple cameras specifically designed to acquire images of specific structural components can be used to acquire image information of multiple structural components of the coal mining machine. Each camera can bind the label information of the structural component it is responsible for acquiring with the image information of that structural component to identify the name of the structural component corresponding to that image information. Subsequently, the image information with tagged information is sent to the control unit 121, which houses the sub-model of the Input network responsible for image acquisition, within the control unit group 120. The sub-model on control unit 121 can stitch together different images by randomly scaling, cropping, and arranging the image information to obtain enhanced image information. Then, the control unit containing the Input network responsible for image acquisition sends the enhanced image information to the control unit 121, which houses the Backbone network sub-model, via bus 110 to obtain a feature map that meets the preset feature size requirements. The control unit 121, which houses the Backbone network sub-model, sends the feature map to the control unit 121, which houses the Neck network sub-model, via bus 110. The control unit 121, which houses the Neck network sub-model, can perform feature fusion on the feature map based on the FPN+PAN structure and send the fused image to the control unit 121, which houses the Head network sub-model, via bus 110. This allows the control unit 121, which houses the Head network sub-model, to output the position information of the structural components in the image, thereby obtaining the working information of the coal mining machine.

[0031] Furthermore, the target detection sub-model is used to control the motion state of its corresponding functional components based on the working information of the coal mining machine. Specifically, the target detection sub-model can be trained using the position information of each structural component of the coal mining machine and the working stage of the coal mining machine as features, and the motion control data of each functional component as labels. This allows the target detection sub-model to control the motion control data of its corresponding functional components based on the working information of the coal mining machine, and to control the operating state of each functional component based on the motion control data.

[0032] Furthermore, the control unit 121 is also equipped with a wireless communication module. This module is used to achieve wireless communication based on a preset heterogeneous networking technology in the event of an abnormal communication between the two control units 121. The heterogeneous networking technology can be a heterogeneous networking technology based on HarmonyOS.

[0033] Furthermore, the control unit 121 is used to determine in real time whether there are any abnormalities in the communication status between the bus 110 and other control units 121 in the control unit group 120. Specifically, it can determine in real time whether the packet loss rate during data transmission and reception with other control units 121 in the control unit group 120 meets the preset communication quality requirements. If the packet loss rate is greater than the preset communication quality requirements, it is determined that there is an abnormality in the communication status with other control units 121 in the control unit group 120. Furthermore, the control unit 121 is also used to activate the wireless communication module (not shown in the figure) set on the control unit 121 and perform data transmission between control units 121 in the control unit group 120 based on a preset heterogeneous network when the communication status is abnormal. The heterogeneous network can be a heterogeneous network using the HarmonyOS soft bus to realize the data continuation of the bus 110 between control units 121 in the fully mechanized mining face in the event of a circuit failure. Specifically, the wireless communication module can be a communication module such as Wi-Fi, 5G, and Bluetooth, and the heterogeneous networking technology of the HarmonyOS system ensures the effective transmission of data between control units 121 after the fully mechanized mining face bus 110 is broken. It can also report circuit breaker information to the remote control center and human-machine interaction system so that staff can take further action on the abnormal situation.

[0034] Furthermore, the aforementioned control unit 121 can also be a control unit 121 in each device that needs to interact with data in the fully mechanized mining face. A bus module and a wireless communication module can be installed on each device that needs to interact with data in the fully mechanized mining face, and corresponding programs can be set on these devices to enable each device to monitor whether its bus communication with other devices is normal. When an abnormality in bus communication is detected, the wireless communication module installed on the device is activated, and data transmission between devices is performed based on a preset heterogeneous network of the HarmonyOS system.

[0035] The working face control system provided in this embodiment can pre-divide the target detection model into multiple sub-models and distribute them among the control units of multiple functional components in the fully mechanized mining face. These control units can interact via a bus to handle the data involved in target detection. Furthermore, the working information of the coal mining machine is obtained through the target detection steps, and the operating status of each functional component is controlled based on this information. Simultaneously, the control units also have the ability to communicate based on heterogeneous networking, improving the system's resilience to abnormal situations such as bus interruptions. This application distributes the computing power required for target detection across multiple control units in the functional components, significantly reducing the computing power requirements of individual control units and thus lowering the hardware cost for achieving autonomous intelligent decision-making in the fully mechanized mining face.

[0036] In one embodiment, the object detection sub-model includes: a raw data acquisition sub-model, a baseline network sub-model, a refined feature extraction sub-model, and a location output sub-model.

[0037] The original data acquisition sub-model receives image information from multiple structural components, binds the image information of each component with its tag information, and sends the tagged image information to the baseline network sub-model. Specifically, it can acquire image information of specific structural components based on cameras responsible for acquiring such information. For example, the control unit group is connected to four cameras that respectively acquire image information of the left rocker arm, the left coal drum, the right rocker arm, and the right coal drum. When the control unit containing the original data acquisition sub-model in the control unit group acquires the image information sent by the camera acquiring the left rocker arm image information, it can bind the tag information of the left rocker arm with that image information. The acquisition and binding methods for image information and tag information of other structural components are similar and will not be repeated here. Furthermore, the control unit containing the original data acquisition sub-model sends the tagged image information to the control unit containing the baseline network sub-model, so that the baseline network sub-model can acquire the tagged image information and perform further processing.

[0038] Furthermore, the baseline network sub-model receives image information bound with label information, performs slicing or convolution operations on the image information to obtain a feature map that meets the preset feature size requirements, and sends the feature map and label information to the thinning feature extraction sub-model. Specifically, the baseline network sub-model can perform convolution operations on the image information and stitch different images together by randomly scaling, cropping, and arranging them to obtain a feature map that meets the preset feature size requirements, such as a 304x304x32 feature map. This feature map is then sent to the control unit where the thinning feature extraction sub-model resides via a bus, so that the thinning feature extraction sub-model can receive the feature map and perform further processing.

[0039] Furthermore, the refined feature extraction sub-model divides the feature map into multiple predicted feature maps at preset feature scales, and sends these predicted feature maps along with label information to the position output sub-model. Specifically, the refined feature extraction sub-model can refine the feature map by extracting features from the obtained feature map, processing the feature map involving structural components to obtain feature maps of different scales, and sending the processed results to the control unit where the position output sub-model resides, so that the position output sub-model can receive feature maps of different scales and perform further processing.

[0040] Furthermore, the location output sub-model identifies the identification anchor frame of the structural component based on predicted feature maps and label information at multiple preset feature scales, and determines the location information of the structural component based on the coordinate information of the identification anchor frame in the image information. Specifically, the location output sub-model can output the identification anchor frame of the structural component based on the feature map, locate the structural component at different scales output by the refined feature extraction sub-model, output the coordinate values ​​of the located structural component of the coal mining machine relative to the image information, and send these coordinates as the location information of the structural component to the bus for subsequent operations.

[0041] The embodiments provided in this application can distribute the computing power required for image recognition to multiple control units, which can significantly reduce the computing power requirements of a single control unit, thereby reducing the hardware cost of the coal mining machine to achieve autonomous intelligent decision-making.

[0042] In one embodiment, the control unit is further configured to convert the data encoding generated and sent out by the control unit into a corresponding object model data encoding based on a preset object model encoding.

[0043] Specifically, a unified device model can be applied to multiple control units, and the data from each control unit can be converted into unified object model data according to the corresponding protocol of object model encoding. This ensures that the object model data is used for application and transmission in all subsequent systems, and the converted values ​​are transmitted using the object model encoding. Furthermore, a unified device model can be applied to multiple devices, and the data from each device can be converted into unified object model data according to the corresponding protocol of object model encoding. This ensures that the object model data is used for application and transmission in all subsequent systems, and the converted values ​​are transmitted using the object model encoding, thereby reducing the cost of inter-system integration. The embodiments provided in this application can reduce the magnitude of data transmission; the unified object model data structure completely eliminates the conversion cost of system data transmission, significantly reducing the cost of inter-system integration.

[0044] The working face control system provided in this embodiment can pre-distribute multiple sub-models of the target detection model into the control units of multiple functional components in the fully mechanized mining face. These control units can interact with each other via a bus. Furthermore, the working information of the coal mining machine is obtained through the target detection step, and the operating status of each functional component is controlled based on this information. Simultaneously, communication between control units can be achieved via the bus when bus communication is normal. If any bus malfunctions or is disconnected, the system can detect the anomaly immediately and utilize the HarmonyOS soft bus heterogeneous networking technology to achieve data continuation between control units within the fully mechanized mining face, ensuring that the intelligent decision-making of the distributed control system is not disturbed and improving the stability of the coal mining machine control. At the same time, the data generated by the control units is standardized according to a protocol, reducing the cost of system integration.

[0045] Furthermore, regarding Figure 1 The above-described working face control system is illustrated in this embodiment, and a working face control method is provided for application in the aforementioned working face control system, such as... Figure 2 As shown, the method includes:

[0046] S201. Real-time acquisition of image information of multiple structural components on the coal mining machine, and input of the image information into a pre-trained target detection model to obtain the position information of each structural component.

[0047] This system can acquire image information of structural components using multiple image acquisition devices mounted on the coal mining machine. Furthermore, each control unit can be a controller for a specific functional component of the fully mechanized mining face, possessing certain computational capabilities, such as a support control unit for controlling hydraulic supports, a scraper conveyor control unit for controlling scraper conveyors, and a spray control unit for controlling spray systems.

[0048] Furthermore, the object detection model can be pre-trained and is capable of determining the location of various structural components in an image based on the input image information. The object detection model comprises multiple sub-models, each executing one of the pre-defined sub-steps of object detection. Each sub-model is housed in a pre-selected control unit, and all control units in the control unit group are equipped with all the sub-models of the object detection model. The control units containing each sub-model can communicate with each other via a data bus, or, in the event of a data bus malfunction, via a wireless network, to collaboratively achieve the object recognition process.

[0049] S202. Determine the position of the coal mining machine in the coal mining face and determine the working stage of the coal mining machine.

[0050] Among them, the coal mining face is the coal mining channel where the coal mining machine carries out coal mining operations, and the working stage of the coal mining machine can include upward stage 1, upward stage 2, upward stage 3, downward stage 1, downward stage 2, and downward stage 3.

[0051] Specifically, the position of the coal mining machine in the coal face can be determined using position sensors or positioning devices installed on the machine. Simultaneously, based on the set coal mining process parameters, the distance the machine needs to travel in each working stage is determined, as well as the total distance traveled since the initialization stage, thus determining the current working stage of the machine. For example, if the total distance traveled by the machine since the initialization stage is 100 meters, and the distances required for upward stage 1, upward stage 2, and upward stage 3 according to the coal mining process parameters are 70 meters, 20 meters, and 20 meters respectively, then the current working stage of the machine can be determined as upward stage 3. Alternatively, it can be directly connected to an external stage determination device to obtain the current working stage of the coal mining machine from an external source.

[0052] S203. Input the location of the coal mining machine, the working stage, the location information of each structural component, and the sensor information collected by at least one sensor set in the fully mechanized mining face into the pre-trained component motion prediction model set in each control unit to obtain the motion control data of each functional component.

[0053] The sensors can include scraper conveyor stroke sensors, coal mining machine speed sensors, support tilt angle sensors, and support pressure sensors installed in the fully mechanized mining face. The scraper conveyor stroke sensor acquires the scraper conveyor's stroke data, the coal mining machine speed sensor acquires the coal mining machine's speed information, the support tilt angle sensor acquires the hydraulic support's tilt angle information, and the support pressure sensor acquires the hydraulic support's pressure information. Furthermore, the target detection sub-model can be pre-trained based on a neural network model. This model uses the position information of each structural component of the coal mining machine, the parameters of the coal mining machine's position information, and the information collected by the aforementioned sensors as features, with the motion control data of each functional component of the coal mining machine trained using labels. Further, the aforementioned component motion prediction models can be set in the control unit of each functional component. This allows each control unit to obtain motion control data for its functional component based on the coal mining machine's position, working stage, the position information of each structural component, and the aforementioned sensor information, through the target detection sub-model. This enables functions such as coal mining machine operation trend prediction, automatic compensation of the pusher stroke, support posture self-adjustment, and support collision prevention.

[0054] Specifically, motion control data can include control data such as pusher stroke compensation values, hydraulic support height and tilt control parameters, scraper conveyor stroke values, and spray range values. This data is used to send motion control data related to each functional component to the corresponding control unit of each functional component, enabling the coal mining machine to make autonomous and intelligent decisions.

[0055] S204. Based on motion control data, control the operating status of each functional component.

[0056] Specifically, each control unit can control the movement of the functional components it is responsible for controlling based on motion control data.

[0057] The working face control method provided in this embodiment can distribute multiple sub-models, based on a pre-defined target detection model, across multiple control units of functional components within the fully mechanized mining face. These control units can interact via a bus to handle data exchange related to target detection. Furthermore, image information of the coal mining machine's structural components, including the left rocker arm, left coal drum, right rocker arm, and right coal drum, can be input into the control unit group. The control units in this group collectively perform target detection, obtaining the position information of each structural component during operation. Then, the coal mining machine's position within the working face and its current working stage are determined. Next, the coal mining machine's position, working stage, and the position information of each structural component, along with sensor information collected by at least one sensor located within the fully mechanized mining face, are input into a pre-trained component motion prediction model located in each control unit. This yields motion control data for each functional component, including hydraulic supports and sprayers. Finally, the operating state of each functional component can be controlled based on the motion control data. This application distributes the computational power required for image recognition across multiple functional components in the control unit, significantly reducing the computational power requirements of individual control units and thus lowering the hardware cost for autonomous intelligent decision-making in fully mechanized mining faces. Furthermore, based on the positional information of each structural component and the working stage of the coal mining machine, motion control data for functional components such as hydraulic supports can be obtained using a component motion prediction model. This allows for precise control of the motion of each functional component, achieving more accurate autonomous intelligent decision-making for the coal mining machine while maintaining lower hardware costs.

[0058] In one embodiment, step 201 can be implemented as follows: First, the original data acquisition sub-model receives image information from multiple structural components, binds the image information of each structural component with its tag information, and sends the image information bound with the tag information to the baseline network sub-model. Specifically, the image information of a specific structural component can be acquired based on a camera responsible for acquiring the image information of that specific structural component. As an example, the control unit group is connected to four cameras that respectively acquire image information of the left rocker arm, the left coal mining drum, the right rocker arm, and the right coal mining drum. When the control unit where the original data acquisition sub-model is located in the control unit group acquires the image information sent by the camera that acquired the image information of the left rocker arm, it can bind the tag information of the left rocker arm with the image information sent by that camera. The acquisition and binding methods of image information and tag information of other structural components are similar to those described above and will not be repeated here. Further, the control unit where the original data acquisition sub-model is located sends the image information bound with the tag information to the control unit where the baseline network sub-model is located, so that the baseline network sub-model can acquire the image information bound with the tag information and perform further processing.

[0059] Then, the baseline network sub-model receives image information bound with label information, performs slicing or convolution operations on the image information to obtain a feature map that meets the preset feature size requirements, and sends the feature map and label information to the thinning feature extraction sub-model. Specifically, the baseline network sub-model can perform convolution operations on the image information and stitch different images together in a random scaling, random cropping, and random arrangement manner to obtain a feature map that meets the preset feature size requirements, such as obtaining a 304x304x32 feature map. This feature map is then sent to the control unit where the thinning feature extraction sub-model resides via a bus, so that the thinning feature extraction sub-model can receive the feature map and perform further processing.

[0060] Next, the refined feature extraction sub-model divides the feature map into multiple predicted feature maps at preset feature scales, and sends these predicted feature maps along with label information to the position output sub-model. Specifically, the refined feature extraction sub-model can refine the feature map by extracting features from the obtained feature map, processing the feature map involving structural components to obtain feature maps of different scales, and sending the processed results to the control unit where the position output sub-model is located, so that the position output sub-model can receive feature maps of different scales and perform further processing.

[0061] Finally, the location output sub-model identifies the identification anchor frame of the structural component based on predicted feature maps at multiple preset feature scales and label information, and determines the location information of the structural component based on the coordinate information of the identification anchor frame in the image information. Specifically, the location output sub-model can output the identification anchor frame of the structural component based on the feature map, locate the structural component at different scales output by the refined feature extraction sub-model, output the coordinates of the located structural component of the coal mining machine relative to the image information, and send these coordinates as the location information of the structural component to the bus for subsequent operations.

[0062] The embodiments provided in this application can distribute the computing power required for target detection to multiple control units, which can significantly reduce the computing power requirements of a single control unit, thereby reducing the hardware cost of the coal mining machine to achieve autonomous and intelligent decision-making.

[0063] In one embodiment, the method for determining the working stage of the coal mining machine in step 202 includes:

[0064] First, the trajectory information of the coal mining machine on the coal mining face is obtained. This trajectory information includes the route the coal mining machine travels on the coal mining face, including the starting point of the coal mining machine in its initial state, the turning points where the coal mining machine changes direction each time, and the direction of travel before and after each turning point.

[0065] Specifically, the starting position of the coal mining machine in its initial state and the turning points where the machine changes direction each time can be obtained based on position sensors installed on the machine. Then, based on the machine's position and trajectory information, along with preset up and down parameters, the working stage is determined. In this process, the up parameters include up parameter 1 and up parameter 2, and the down parameters include down parameter 1 and down parameter 2. Further, the value of up parameter 1 can be the number of a preset first hydraulic support within the working face, indicating that when the machine reaches the preset hydraulic support, it turns back, and the working stage transitions from up stage 1 to up stage 2. The value of up parameter 2 can be the number of a preset second hydraulic support within the working face, indicating that when the machine moves from the first hydraulic support to the second hydraulic support, it turns back, and the working stage transitions from up stage 2 to up stage 3. When the coal mining machine is in the upward stage 3, and moves to the first hydraulic support again, the coal mining machine turns back, and its working stage transitions from the upward stage 3 to the downward stage 1.

[0066] Furthermore, the value of the downshift parameter 1 can be the number of the third hydraulic support pre-set within the coal mining face, indicating that when the coal mining machine in downshift stage 1 reaches the position of the third hydraulic support, the coal mining machine turns back, and the working stage transitions from downshift stage 1 to downshift stage 2. The value of the downshift parameter 2 can be the number of the fourth hydraulic support pre-set within the coal mining face, indicating that when the coal mining machine in downshift stage 3 reaches the position of the fourth hydraulic support, the coal mining machine turns back, and the working stage transitions from downshift stage 2 to downshift stage 3. When the coal mining machine, in downshift stage 3, moves back to the position of the third hydraulic support, the coal mining machine turns back, and its working stage transitions from downshift stage 3 to upshift stage 1. The values ​​of upshift stage 1, upshift stage 2, downshift stage 1, and downshift stage 2 can be determined based on actual conditions.

[0067] Then, based on the position and trajectory information of the coal mining machine and preset up and down parameters, the working stage is determined. Specifically, the working stage can be determined by comparing the hydraulic support numbers corresponding to the two most recent turnaround points of the coal mining machine before the current time with the hydraulic support numbers recorded in the preset up and down parameters. If they match, the direction of the coal mining machine before and after the turnaround is obtained, thereby determining the current working stage of the coal mining machine. The embodiments provided in this application can accurately determine the current working stage of the coal mining machine based on its operating trajectory and preset process parameters, providing a basis for subsequent control work.

[0068] In one embodiment, the method for obtaining the trajectory information of the coal mining machine at the coal mining face includes:

[0069] First, during the operation of the coal mining machine within the coal face, the positions of the machine during the previous preset number of directional turnarounds are determined in ascending order of time intervals from the current time. The preset number of turnarounds can be set to two, with the specific number determined based on actual conditions. For example, when the coal mining machine is at its most recent turnaround point, its operating direction changes from moving from the higher-numbered hydraulic support towards the lower-numbered hydraulic support to moving towards the higher-numbered hydraulic support. Similarly, when the coal mining machine is at its second closest turnaround point, its operating direction changes from moving from the lower-numbered hydraulic support towards the higher-numbered hydraulic support to moving towards the lower-numbered hydraulic support. This allows us to determine the positions of the coal mining machine at the two most recent turnarounds and the operating direction before and after the turnarounds.

[0070] Then, the support number corresponding to each location is determined. Specifically, based on the installation position and support number of each hydraulic support within the coal mining face, the support number corresponding to the location where the coal mining machine turns back is determined. Finally, based on the support numbers, the trajectory information of the coal mining machine on the coal mining face is determined. Specifically, based on the support numbers of the hydraulic supports corresponding to the positions of the coal mining machine during its two most recent turns, and considering the running direction before and after each turn, the running trajectory of the coal mining machine within the coal mining face can be derived. In the embodiments of this application, the trajectory information of the coal mining machine can be accurately determined based on the direction and position of the coal mining machine during its most recent turns, providing a basis for subsequently determining the working stage of the coal mining machine.

[0071] In one embodiment, the working face control method further includes: first, monitoring the status of heartbeat signals transmitted between any two control units connected via a bus. Specifically, multiple control units in a control unit group can be divided into two groups, and the control units within each group monitor the heartbeat signals sent by another control unit within the group via the bus in real time. Then, if the status of the heartbeat signal is abnormal, it is determined that the communication status is abnormal. Specifically, if the control unit within the group detects an interruption in the heartbeat signal sent by another control unit within the group via the bus, it can be determined that the communication status is abnormal. Further, in the case of abnormal communication status, the wireless communication module set on the control unit is activated, and data transmission between adjacent control units is performed based on a preset heterogeneous networking technology. In the embodiments of this application, when a bus connection is abnormal, the occurrence of the abnormality can be determined immediately, so as to achieve emergency handling in the shortest possible time. Further, a corresponding program can be set on each device in the fully mechanized mining face that needs to perform data interaction, so that the device can monitor the status of the heartbeat signals transmitted via the bus between the device and other devices with which it needs to communicate. If the device detects that the status of the heartbeat signal is abnormal, it can be determined that there is an abnormality in the bus communication with other devices.

[0072] The working face control method provided in this embodiment distributes the computing power required for target detection across multiple functional components in the control unit, significantly reducing the computing power requirements of individual control units and thus lowering the hardware cost for autonomous intelligent decision-making in fully mechanized mining faces. Furthermore, based on the positional information of each structural component and the working stage of the coal mining machine, motion control data for functional components such as hydraulic supports can be obtained using a component motion prediction model. This allows for precise control of the motion of each functional component. The basic AI module predicts the movement trend of the coal mining machine, enabling the scraper conveyor controller to perform push-conveyor movement based on the predicted trend and collected scraper conveyor stroke data, forming an automatic compensation mechanism. The support controller intelligently adjusts its posture based on collected data such as inclination angle and pressure to meet parameter model settings, and further adjusts its actions according to the posture of the coal mining machine to prevent collisions. This application can achieve more accurate autonomous intelligent decision-making in fully mechanized mining faces while maintaining low hardware costs.

[0073] Furthermore, as Figure 2 The specific implementation of the method shown in this embodiment provides a working surface control device, such as... Figure 3 As shown, the device includes: a position recognition module 31, a stage determination module 32, a data generation module 33, and a motion control module 34.

[0074] The position recognition module 31 can be used to acquire image information of multiple structural components on the coal mining machine in real time, and input the image information into a pre-trained target detection model to obtain the position information of each structural component.

[0075] The stage determination module 32 can be used to determine the position of the coal mining machine in the coal mining face and to determine the working stage of the coal mining machine.

[0076] The data generation module 33 can be used to input the position of the coal mining machine, the working stage, the position information of each structural component, and sensor information collected by at least one sensor set in the fully mechanized mining face into a pre-trained component motion prediction model set in each control unit to obtain motion control data for each functional component. The motion control module 34 can be used to control the operating state of each functional component based on the motion control data.

[0077] In a specific application scenario, the location recognition module 31 can be used to: receive image information of multiple structural components from the original data acquisition sub-model; bind the image information of each structural component with its label information; and send the image information bound with the label information to the baseline network sub-model. The baseline network sub-model receives the image information bound with the label information, performs slicing or convolution operations on the image information to obtain a feature map that meets the preset feature size requirements, and sends the feature map and the label information to the refinement feature extraction sub-model. The refinement feature extraction sub-model divides the feature map into multiple predicted feature maps of preset feature scales, and sends the multiple predicted feature maps of preset feature scales and the label information to the location output sub-model. The location output sub-model identifies the recognition anchor map of the structural component based on the multiple predicted feature maps of preset feature scales and the label information, and determines the location information of the structural component based on the coordinate information of the recognition anchor map in the image information.

[0078] In a specific application scenario, the stage determination module 32 can be used to obtain the trajectory information of the coal mining machine on the coal mining face; and determine the working stage based on the position of the coal mining machine and the trajectory information, as well as preset uplink parameters and preset downlink parameters.

[0079] In specific application scenarios, the stage determination module 32 can also be used to determine the location of the coal mining machine during the operation of the coal mining machine in the coal mining face, in ascending order of time intervals from the current time, when the machine turns back a preset number of times; determine the support number of the support corresponding to each location; and determine the trajectory information of the coal mining machine in the coal mining face based on the support number.

[0080] Furthermore, such as Figure 4 As shown, the working face control device also includes a status recognition module 45.

[0081] The status recognition module 45 can be used to monitor the status of the heartbeat signal transmitted between any two control units connected by a bus; if the status of the heartbeat signal is abnormal, it is determined that the communication status is abnormal; in the case of abnormal communication status, the wireless communication module set on the control unit is activated and data transmission between adjacent control units is performed based on a preset heterogeneous networking technology.

[0082] It should be noted that other corresponding descriptions of the functional units involved in the working face control device provided in this embodiment can be found in [reference needed]. Figure 2 The corresponding description in [the document] will not be repeated here.

[0083] Based on the above, Figure 2 Accordingly, this embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the above-described method. Figure 2 The working face control method shown.

[0084] Based on this understanding, the technical solution of this application can be embodied in the form of a software product. The software product to be identified can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive), including several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods described in the various implementation scenarios of this application.

[0085] Based on the above, Figure 2 The method shown, and Figure 3 , Figure 4 To achieve the above objectives, the illustrated work surface control device embodiment also provides a physical device for work surface control. Specifically, this device can be a personal computer, server, smartphone, tablet computer, smartwatch, or other network device. This physical device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to achieve the above-described objectives. Figure 2 The method shown.

[0086] Optionally, the physical device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0087] Those skilled in the art will understand that the physical device structure for working surface control provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0088] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs to be identified. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms, or it can be implemented by hardware. By applying the technical solution of this application, firstly, image information of multiple structural components on the coal mining machine is acquired in real time, and the image information is input into a pre-trained target detection model to obtain the position information of each structural component; then, the position of the coal mining machine in the coal mining face is determined, and the working stage of the coal mining machine is determined; then, the position of the coal mining machine, the working stage, the position information of each structural component, and the sensor information collected by at least one sensor set in the fully mechanized mining face are respectively input into a pre-trained component motion prediction model set in each control unit to obtain motion control data of each functional component; finally, based on the motion control data, the operating state of each functional component is controlled. Compared with the prior art, the computing power required for image recognition can be distributed among multiple control units in the coal mining machine, which can significantly reduce the computing power requirements of a single control unit, thereby reducing the hardware cost of the coal mining machine to achieve autonomous intelligent decision-making. Furthermore, based on the positional information of each structural component and the working stage of the coal mining machine, motion control data for functional components such as hydraulic supports can be obtained based on the component motion prediction model. This allows for precise control of the motion of each functional component, enabling more accurate autonomous and intelligent decision-making of the coal mining machine while maintaining low hardware costs.

[0090] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be modified to be located in one or more apparatuses different from this embodiment. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0091] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A working face control system, characterized in that, It includes a control unit group consisting of multiple control units, which are respectively disposed on multiple functional components on the fully mechanized mining face, and the multiple control units are connected based on a bus; The control unit group is equipped with a target detection model, and each control unit is equipped with a different target detection sub-model. The target detection sub-model is used to execute one of the multiple sub-steps pre-divided based on the target detection steps. The target detection model is used to execute the target detection steps and obtain the working information of the coal mining machine. The target detection sub-model is used to control the movement state of the corresponding functional components based on the working information of the coal mining machine. The target detection sub-models include: a raw data acquisition sub-model, a baseline network sub-model, a refined feature extraction sub-model, and a location output sub-model; The original data acquisition sub-model is used to receive image information of multiple structural components of the coal mining machine, bind the image information of each structural component with the tag information of the structural component, and send the image information bound with the tag information to the baseline network sub-model; The baseline network sub-model is used to receive the image information bound with the label information, perform slicing or convolution operations on the image information to obtain a feature map that meets the preset feature size requirements, and send the feature map and the label information to the thinning feature extraction sub-model; The refined feature extraction sub-model is used to divide the feature map into multiple predicted feature maps of preset feature scales, and send the multiple predicted feature maps of preset feature scales and the label information to the position output sub-model; The location output sub-model is used to identify the identification anchor frame of the structural component based on the predicted feature map of multiple preset feature scales and the label information, and to determine the location information of the structural component based on the coordinate information of the identification anchor frame in the image information; The control unit is also equipped with a wireless communication module, which is used to achieve wireless communication based on a preset heterogeneous networking technology when the communication between the two control units is abnormal. The working face control system is configured to perform the following processes: The image information of multiple structural components on the coal mining machine is acquired in real time, and the image information is input into a pre-trained target detection model to obtain the position information of each structural component; The method involves determining the position of the coal mining machine within the coal mining face, and then, during the operation of the coal mining machine within the coal mining face, acquiring the position of the coal mining machine during the previous preset number of directional reversals in ascending order of time intervals from the current time. The method further involves determining the support number corresponding to each position, determining the trajectory information of the coal mining machine within the coal mining face based on the support number, and determining the working stage based on the coal mining machine position, the trajectory information, and preset up and down parameters. The location of the coal mining machine, the working stage, the location information of each structural component, and the sensor information collected by at least one sensor set in the fully mechanized mining face are respectively input into the pre-trained component motion prediction model set in each control unit to obtain the motion control data of each functional component. Based on the motion control data, the operating state of each of the functional components is controlled.

2. The working face control system according to claim 1, characterized in that, The control unit is also used for: Based on the preset object model code, the data code generated and sent out by the control unit is converted into an object model data code corresponding to the object model code.

3. A working face control method, characterized in that, The control method is applied to the working face control system as described in any one of claims 1-2, and the method includes: The raw data acquisition sub-model receives image information of multiple structural components, binds the image information of each structural component with the label information of the structural component, and sends the image information bound with the label information to the baseline network sub-model; The baseline network sub-model receives the image information bound with the label information, performs slicing or convolution operations on the image information to obtain a feature map that meets the preset feature size requirements, and sends the feature map and the label information to the thinning feature extraction sub-model; The refined feature extraction sub-model divides the feature map into multiple predicted feature maps of preset feature scales, and sends the multiple predicted feature maps of preset feature scales and the label information to the position output sub-model; The location output sub-model identifies the identification anchor map of the structural component based on the predicted feature map of the multiple preset feature scales and the label information, and determines the location information of the structural component based on the coordinate information of the identification anchor map in the image information.

4. The method according to claim 3, characterized in that, The method further includes: Monitor the status of the heartbeat signal transmitted between any two control units connected via a bus; If the state of the heartbeat signal is abnormal, then the communication status is determined to be abnormal; In the event of an abnormal communication condition, the wireless communication module installed on the control unit is activated and data transmission between adjacent control units is performed based on a preset heterogeneous networking technology.

5. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 3 to 4.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 3 to 4.

Images