Coal conveyor corridor multi-modal inspection method, device, equipment and readable storage medium

Abstract

The present invention provides a multimodal inspection method, device, equipment and readable storage medium for coal transportation corridors, which relates to the technical field of coal transportation corridors, including obtaining first information and second information; using image information in the first information as input and fault adjustment information as output, training and testing a preset prediction model to obtain a fault adjustment prediction model; constructing a scoring function based on the fault adjustment prediction model and the second information; obtaining the monitoring results of the target coal transportation corridor based on the real-time image information, real-time key monitoring parameters and scoring function of the target coal transportation corridor, and performing fault warning according to the monitoring results. The present invention uses image information and key monitoring parameters to comprehensively integrate multiple monitoring signals to provide more comprehensive real-time monitoring for coal transportation corridors. The combination of multimodal information avoids omissions or misjudgments that may be caused by a single data source, thereby improving the automation and real-time nature of fault detection.

Description

Technical Field

[0001] The present invention relates to the technical field of coal transportation corridors, and in particular to a multi-modal inspection method, device, equipment and readable storage medium for coal transportation corridors. Background Art

[0002] The traditional coal corridor monitoring system relies on manual inspection and image detection, which has significant limitations. Due to manpower and time constraints, manual inspections are difficult to achieve continuous and comprehensive monitoring, and regular inspections are only carried out at fixed time intervals, which cannot respond to sudden failures in a timely manner and easily miss critical moments. In addition, image technology can capture the appearance of equipment, but it is interfered by environmental factors such as lighting and angle, and it is difficult to detect hidden faults inside the equipment, such as component wear or electrical problems. Image recognition cannot accurately capture subtle or non-visually visible problems, which may lead to misjudgment or omission, affecting the accuracy of fault detection. Therefore, the comprehensive use of image data and other monitoring parameters, combined with intelligent algorithms such as deep learning for analysis, can improve the accuracy, real-time and security of the monitoring system and avoid the shortcomings of traditional methods. Summary of the invention

[0003] The purpose of the present invention is to provide a multi-modal inspection method, device, equipment and readable storage medium for coal transportation corridors to improve the above problems. In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0004] In a first aspect, the present application provides a multi-modal inspection method for a coal transportation corridor, comprising:

[0005] Acquire first information and second information, wherein the first information is image information of the coal transportation corridor and corresponding fault adjustment information, and the second information is key monitoring parameters affecting the fault adjustment information;

[0006] As a unit, used to take the image information in the first information as input and the fault adjustment information as output, train and test a preset prediction model to obtain a fault adjustment prediction model, wherein the preset prediction model is a recurrent neural network model based on KERAS;

[0007] constructing a scoring function based on the fault adjustment prediction model and the second information;

[0008] Based on the real-time image information of the target coal transportation corridor, the real-time key monitoring parameters and the scoring function, the monitoring result of the target coal transportation corridor is obtained, and a fault warning is performed according to the monitoring result.

[0009] In a second aspect, the present application also provides a multi-modal inspection device for a coal conveying corridor, comprising:

[0010] A first acquisition unit is used to acquire first information and second information, wherein the first information is image information of the coal transportation corridor and corresponding fault adjustment information, and the second information is a key monitoring parameter that affects the fault adjustment information;

[0011] As a unit, used to take the image information in the first information as input and the fault adjustment information as output, train and test a preset prediction model to obtain a fault adjustment prediction model, wherein the preset prediction model is a recurrent neural network model based on KERAS;

[0012] A first construction unit, configured to construct a scoring function based on the fault adjustment prediction model and the second information;

[0013] The first obtaining unit is used to obtain the monitoring result of the target coal transportation corridor based on the real-time image information of the target coal transportation corridor, the real-time key monitoring parameters and the scoring function, and to perform fault warning according to the monitoring result.

[0014] In a third aspect, the present application also provides a multi-modal inspection device for a coal transportation corridor, including:

[0015] Memory for storing computer programs;

[0016] A processor is used to implement the steps of the multimodal inspection method of the coal transportation corridor when executing the computer program.

[0017] In a fourth aspect, the present application further provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned multimodal inspection method for coal transportation corridors are implemented.

[0018] The beneficial effects of the present invention are:

[0019] The present invention integrates multiple monitoring signals through image information and key monitoring parameters to provide more comprehensive real-time monitoring for the coal transportation corridor. The combination of multimodal information avoids omissions or misjudgments that may be caused by a single data source, thereby improving the automation and real-time performance of fault detection.

[0020] Other features and advantages of the present invention will be set forth in the following description, and in part will become apparent from the description, or may be understood by practicing embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram of a multi-modal inspection method for a coal transportation corridor according to an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of prediction logic of the prediction model described in an embodiment of the present invention;

[0024] Figure 3 It is a schematic structural diagram of a multi-modal inspection device for a coal conveying corridor according to an embodiment of the present invention;

[0025] Figure 4 It is a schematic diagram of the structure of the multi-modal inspection equipment for coal transportation corridors described in an embodiment of the present invention.

[0026] Markings in the figure: 10, first acquisition unit; 20, as unit; 30, first construction unit; 40, first obtaining unit; 800, multimodal inspection equipment of coal transportation corridor; 801, processor; 802, memory; 803, multimedia component; 804, I / O interface; 805, communication component. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0028] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. Embodiment 1:

[0029] This embodiment provides a multi-modal inspection method for coal transportation corridors.

[0030] See also Figure 1 , the figure shows that the method includes step S10, step S20, step S30 and step S40.

[0031] Step S10. Acquire first information and second information, the first information being image information of the coal transportation corridor and corresponding fault adjustment information, and the second information being key monitoring parameters affecting the fault adjustment information;

[0032] Specifically, the image information of the coal conveyor corridor in the first information can be obtained through a camera set in the coal conveyor corridor or a camera on a patrol robot. The fault adjustment information includes the adjustment intensity of each adjustment measure. The adjustment measure refers to the adjustment of various operations or equipment to solve the fault when there is a fault in the coal conveyor corridor. For example, when problems such as overtemperature, coal powder accumulation, and abnormal vibration occur, different adjustment measures can be used, such as increasing ventilation or reducing the conveyor belt speed to solve these problems; the adjustment intensity indicates the degree of influence of these measures on the operating status of the equipment or system during the adjustment process, for example, adjusting the gear of the ventilation fan by several gears, or reducing the conveyor belt speed by several gears.

[0033] Considering that the sensors in the coal transportation corridor usually include temperature sensors, humidity sensors, coal powder concentration sensors, vibration sensors, pressure sensors, noise sensors, coal dust concentration sensors, wind speed and direction sensors, and temperature sensors, etc., the data collected by these sensors have different effects on the monitoring and regulation of corridor operation. In order to avoid information overload, this application mainly considers sensor data that has a greater impact on corridor operation, and other sensor data that has a smaller impact on corridor operation is not considered, thereby reducing the amount of calculation and improving the efficiency and accuracy of the calculation.

[0034] Specifically, step S10 specifically includes step S11, step S12 and step S13:

[0035] Step S11. Acquire monitoring parameters of the coal transportation corridor, where the monitoring parameters are all parameters collected by sensors arranged in the coal transportation corridor;

[0036] Step S12. Simplify the monitoring parameters by using a rough set reduction algorithm to obtain the monitoring parameters to be screened;

[0037] Step S13. Calculate the importance of the monitoring parameters to be screened to the fault adjustment information through the random forest algorithm, and determine the key monitoring parameters according to the importance;

[0038] Specifically, in this application, the key monitoring parameters screened out through the above steps are noise, temperature, ammonia content and coal dust content as key monitoring parameters. There may be differences in the key monitoring parameters corresponding to different considerations, and no special restrictions are made here.

[0039] Step S20. Taking the image information in the first information as input and the fault adjustment information as output, training and testing a preset prediction model to obtain a fault adjustment prediction model, wherein the preset prediction model is a recurrent neural network model based on KERAS;

[0040] Specifically, step S20 includes step S21, step S22, step S23 and step S24:

[0041] Step S21. Calculate the contribution of image features to the occurrence of faults based on the random forest algorithm, where the image features are features related to various faults on the image of the coal transportation corridor;

[0042] Step S22. Calculate the decisiveness of the fault occurrence on the fault adjustment information based on the random forest algorithm;

[0043] Step S23. Construct a recurrent neural network model based on the KERAS deep learning framework;

[0044] Step S24. Setting the weight matrix of the recurrent neural network model based on the matrix calculation rule according to the contribution and the determinacy to obtain a prediction model;

[0045] Specifically, by combining the random forest algorithm with the recurrent neural network, the image characteristics and fault adjustment information of the coal transportation corridor can be comprehensively considered, so as to accurately realize the prediction and adjustment of faults.

[0046] Specifically, step S24 includes step S241, step S242 and step S243:

[0047] Step S241. Based on the contribution of each image feature to the occurrence of each fault, a first matrix is ​​constructed based on a matrix calculation rule;

[0048] Step S242. Based on the determinism of each fault occurrence on each fault adjustment information, a second matrix is ​​constructed based on a matrix calculation rule;

[0049] Step S243. Use the first matrix and the second matrix as weight matrices of the hidden layer and the output layer of the recurrent neural network model respectively to obtain a prediction model;

[0050] Specifically, the present application takes into account all the regulating measures of the coal transportation corridor, as well as the possible causes of failures corresponding to each regulating measure, and the image features that may exist when the failure occurs, so as to construct the prediction model in the present application.

[0051] The adjustment measures considered in this application include: conveyor belt adjustment, ventilation adjustment, equipment adjustment, dust removal adjustment, cleaning system adjustment, load adjustment, cooling system adjustment, coal powder temperature adjustment, coal powder mixing ratio adjustment and electrical equipment adjustment.

[0052] In this application, the failures are mainly divided into: coal powder blockage, conveyor belt damage, equipment failure, coal dust accumulation, abnormal temperature, the presence of foreign matter, coal powder quality problems, abnormal ventilation, coal conveyor corridor damage and electrical equipment abnormality.

[0053] In this application, the image features are mainly divided into coal powder features, coal powder distribution features, conveyor belt features, motion state features, parts features, clarity features, temperature features and ventilation features.

[0054] Let the first matrix be , the second matrix is ​​set to The contribution of coal powder characteristics, coal powder distribution characteristics, conveyor belt characteristics, motion state characteristics, parts characteristics, clarity characteristics, temperature characteristics and ventilation characteristics to coal powder blockage is as follows: The contribution of coal powder characteristics, coal powder distribution characteristics, conveyor belt characteristics, motion state characteristics, parts characteristics, clarity characteristics, temperature characteristics and ventilation characteristics to conveyor belt damage is as follows: The contribution of coal powder characteristics, coal powder distribution characteristics, conveyor belt characteristics, motion state characteristics, parts characteristics, clarity characteristics, temperature characteristics and ventilation characteristics to equipment failure is as follows: The contribution of coal powder characteristics, coal powder distribution characteristics, conveyor belt characteristics, motion state characteristics, parts characteristics, clarity characteristics, temperature characteristics and ventilation characteristics to coal dust accumulation is as follows: The contribution of coal powder characteristics, coal powder distribution characteristics, conveyor belt characteristics, motion state characteristics, parts characteristics, clarity characteristics, temperature characteristics and ventilation characteristics to temperature anomaly is as follows: The contribution of coal powder characteristics, coal powder distribution characteristics, conveyor belt characteristics, motion state characteristics, parts characteristics, clarity characteristics, temperature characteristics and ventilation characteristics to the appearance of foreign matter is as follows: The contribution of coal powder characteristics, coal powder distribution characteristics, conveyor belt characteristics, motion state characteristics, parts characteristics, clarity characteristics, temperature characteristics and ventilation characteristics to the coal powder quality problem is as follows: The contribution of coal powder characteristics, coal powder distribution characteristics, conveyor belt characteristics, motion state characteristics, parts characteristics, clarity characteristics, temperature characteristics and ventilation characteristics to ventilation abnormality is as follows: The contribution of coal powder characteristics, coal powder distribution characteristics, conveyor belt characteristics, motion state characteristics, parts characteristics, clarity characteristics, temperature characteristics and ventilation characteristics to the damage of coal conveying corridor is as follows: The contribution of coal powder characteristics, coal powder distribution characteristics, conveyor belt characteristics, motion state characteristics, parts characteristics, clarity characteristics, temperature characteristics and ventilation characteristics to the abnormality of electrical equipment is as follows: ;

[0055] Construct the first matrix according to the matrix calculation rules :

[0056]

[0057] The first matrix has eight rows and ten columns, the eight rows correspond to eight image features, and the ten columns correspond to ten fault occurrences.

[0058] The decisive factors for the adjustment of the conveyor belt are coal powder blockage, conveyor belt damage, equipment failure, coal dust accumulation, abnormal temperature, foreign matter, coal powder quality problems, abnormal ventilation, coal conveyor corridor damage and electrical equipment abnormality. The decisive factors for ventilation regulation are coal powder blockage, conveyor belt damage, equipment failure, coal dust accumulation, abnormal temperature, foreign matter, coal powder quality problems, ventilation abnormality, coal conveyor corridor damage and electrical equipment abnormality. The decisive factors for equipment adjustment are coal powder blockage, conveyor belt damage, equipment failure, coal dust accumulation, abnormal temperature, foreign matter, coal powder quality problems, abnormal ventilation, coal conveyor corridor damage and electrical equipment abnormality. The decisive factors for dust removal adjustment are coal dust blockage, conveyor belt damage, equipment failure, coal dust accumulation, abnormal temperature, foreign matter, coal dust quality problems, abnormal ventilation, coal conveyor corridor damage and electrical equipment abnormality. The decisive factors for the adjustment of the cleaning system are coal dust blockage, conveyor belt damage, equipment failure, coal dust accumulation, abnormal temperature, foreign matter, coal dust quality problems, abnormal ventilation, coal conveyor corridor damage and electrical equipment abnormality. The decisive factors for load regulation are coal powder blockage, conveyor belt damage, equipment failure, coal dust accumulation, temperature abnormality, foreign matter, coal powder quality problems, ventilation abnormality, coal conveyor corridor damage and electrical equipment abnormality. The decisive factors for the adjustment of the cooling system are coal powder blockage, conveyor belt damage, equipment failure, coal dust accumulation, abnormal temperature, foreign matter, coal powder quality problems, ventilation abnormality, coal conveyor corridor damage and electrical equipment abnormality. ;

[0059] The decisive factors for the temperature regulation of coal powder are coal pulverized blockage, conveyor belt damage, equipment failure, coal dust accumulation, abnormal temperature, foreign matter, coal pulverized quality problems, ventilation abnormality, coal conveyor corridor damage and electrical equipment abnormality. The decisive factors for adjusting the coal powder mixing ratio are coal powder blockage, conveyor belt damage, equipment failure, coal dust accumulation, abnormal temperature, foreign matter, coal powder quality problems, abnormal ventilation, coal conveyor corridor damage and electrical equipment abnormality. The decisive factors for the regulation of electrical equipment are coal dust blockage, conveyor belt damage, equipment failure, coal dust accumulation, abnormal temperature, foreign matter, coal dust quality problems, abnormal ventilation, coal conveyor corridor damage and electrical equipment abnormality. .

[0060] Construct the second matrix according to the matrix calculation rules :

[0061]

[0062] The second matrix has ten rows and ten columns, the ten rows correspond to ten faults, and the ten columns correspond to ten adjustment measures.

[0063] like Figure 2 As shown, the current moment is The input of the moment is combined with the previous moment. The memory of the moment is then combined with the first matrix of the hidden layer Multiply to get The memory of the moment, through The memory of the moment and the second matrix of the output layer Multiply to get The predicted value at the next moment Input combination at the moment The memory of the moment is then combined with the first matrix of the hidden layer Multiply to get The memory of the moment, through The memory of the moment and the second matrix of the output layer Multiply to get The predicted value at the moment, and so on. Considering that the recurrent neural network model has multiple inputs and leads to multiple outputs, the time step of the prediction model is set as required, and the predicted value at the last moment in the time step is taken as the final output result.

[0064] By setting the hidden layer of the recurrent neural network model through contribution, the influence relationship between image features and fault occurrence is reflected in the hidden layer as the input of the output layer, thereby improving the accuracy of the prediction results. By setting the output layer of the recurrent neural network model deterministically, the influence relationship between each fault occurrence and fault adjustment information is reflected in the output layer of the recurrent neural network model that generates the output, thereby further improving the accuracy of the prediction results.

[0065] Step S30. Constructing a scoring function based on the fault adjustment prediction model and the second information;

[0066] Specifically, the focus of the prior art is to determine the corresponding fault adjustment plan based on the information collected in the current coal transportation corridor, so as to solve potential faults and ensure the normal operation of coal transportation. In this application, after determining the corresponding fault adjustment plan based on the image information of the coal transportation corridor, the current coal transportation corridor will be scored according to the fault adjustment plan and other parameters in the coal transportation corridor. The higher the score, the lower the level of the possible fault problem in the current coal transportation corridor, and it can be adjusted directly according to the adjustment plan; the lower the score, the higher the level of the possible fault problem in the current coal transportation corridor, and the corresponding fault adjustment plan needs to be sent to the staff for review first, so as to ensure that the handling of complex faults will not be too hasty. At the same time, the staff can conduct in-depth analysis according to the actual situation, and can revise the existing fault adjustment plan. After the review is passed, the adjustment plan will be executed to ensure the safety of the operation.

[0067] Specifically, step S30 specifically includes step S31, step S32, step S33, step S34 and step S35:

[0068] Step S31. Calculate a frequency score based on each key monitoring parameter and a preset range of each key monitoring parameter to obtain a plurality of first scores;

[0069] Specifically, by calculating the degree of deviation of each key monitoring parameter from its preset range, a score value based on its degree of compliance can be assigned to each key monitoring parameter, and a lower score is given if the deviation is large.

[0070] Specifically, step S31 specifically includes step S311, step S312, step S313, step S314, step S315 and step S316:

[0071] Step S311. Compare the key monitoring parameters with the corresponding preset ranges to obtain comparison results;

[0072] Step S312. When the comparison result indicates that the key monitoring parameter is within the preset range, the first score value is set to a set threshold;

[0073] Step S313. When the comparison result indicates that the key monitoring parameter is not within the preset range, the difference between the key monitoring parameter and the lower limit of the preset range is calculated to obtain a first value;

[0074] Step S314. Calculate the difference between the upper limit value and the lower limit value of the preset range to obtain a second value;

[0075] Step S315. Calculate the difference between the first value and the second value to obtain a deviation range;

[0076] Step S316. Calculate an exponential function value with the natural constant as the base and the opposite number of the deviation range as the exponent as the first score;

[0077] Specifically, key monitoring parameters include noise ,temperature , Ammonia content and coal dust content When the key monitoring parameter is within the preset range, the first score is 1; when the key monitoring parameter is not within the preset range, the corresponding first score calculation formula is:

[0078] ;

[0079] in, For the First score of key monitoring parameters; For the Key monitoring parameters: For the The upper limit of the preset range of a key monitoring parameter; For the The lower limit of the preset range of a key monitoring parameter; .

[0080] Step S32. Calculate the adjustment score based on the adjustment strength of each adjustment measure and the average adjustment strength of each adjustment measure to obtain a plurality of second scores;

[0081] Specifically, step S32 specifically includes step S321, step S322 and step S323:

[0082] Step S321. Calculate the difference between the regulation intensity of the regulation measure and the corresponding average regulation intensity to obtain a third value;

[0083] Step S322. Calculate the ratio of the third value to the average adjustment intensity to obtain a fourth value;

[0084] Step S323. Calculate an exponential function value with the natural constant as the base and the opposite number of the fourth value as the exponent to obtain a second score;

[0085] Specifically, the adjustment measures include conveyor belt adjustment , ventilation adjustment , Equipment Adjustment , Dust removal adjustment , Cleaning system adjustment , Load Regulation , Cooling system adjustment , Pulverized coal temperature regulation , Coal powder mixing ratio adjustment and electrical equipment regulation , the second score calculation formula is:

[0086] ;

[0087] in, For the a second score for each moderator; For the the intensity of each adjustment measure; For the The average moderating strength of the moderating measures.

[0088] Step S33. Calculate the product of the weight coefficient corresponding to each key monitoring parameter and the first score to obtain multiple first products;

[0089] Step S34. Calculate the product of the weight coefficient corresponding to each adjustment measure and the second score to obtain multiple second products;

[0090] Step S35. Calculate the sum of the multiple first scores and the multiple second scores to obtain a scoring function;

[0091] Specifically, the scoring function calculation formula is:

[0092] ;

[0093] in, is the scoring function value; For the The weight coefficient of each key monitoring parameter; For the First score of key monitoring parameters; For the The weight coefficient of each adjustment measure; For the A second score for each adjustment measure.

[0094] Step S40. Based on the real-time image information, real-time key monitoring parameters and scoring function of the target coal transportation corridor, the monitoring results of the target coal transportation corridor are obtained, and fault warning is performed according to the monitoring results;

[0095] Specifically, step S40 specifically includes step S41, step S42 and step S43:

[0096] Step S41. Calculate a real-time scoring function value based on real-time image information and real-time key monitoring parameters;

[0097] Step S42: Map the scoring function value to a score range between 0 and 100 to obtain the safety quality score of the target coal transportation corridor;

[0098] Step S43. Obtain a fault warning based on the preset safety quality scoring standard and safety quality score;

[0099] Specifically, the safety quality scoring function is:

[0100] ;

[0101] in, is the scoring function value; is the minimum scoring function value; is the maximum scoring function value;

[0102] The minimum score function value indicates the score corresponding to the highest level of fault problems, and the maximum score function value indicates the score corresponding to the lowest level of fault problems. This setting can make the safety quality score of the coal transportation corridor have a unified scale, which is convenient for understanding and comparing the safety quality of coal transportation corridors under different circumstances. Therefore, according to the safety quality score and the preset safety quality score standard, it is determined whether the current adjustment plan needs to be sent to the staff for confirmation. Embodiment 2:

[0103] like Figure 3 As shown, this embodiment provides a multi-modal inspection device for a coal conveying corridor, the device comprising:

[0104] A first acquisition unit 10 is used to acquire first information and second information, the first information being image information of the coal conveying corridor and corresponding fault adjustment information, and the second information being key monitoring parameters affecting the fault adjustment information;

[0105] As a unit 20, it is used to take the image information in the first information as input and the fault adjustment information as output, train and test a preset prediction model to obtain a fault adjustment prediction model, wherein the preset prediction model is a recurrent neural network model based on KERAS;

[0106] A first construction unit 30, configured to construct a scoring function based on the fault adjustment prediction model and the second information;

[0107] The first obtaining unit 40 is used to obtain the monitoring result of the target coal transportation corridor based on the real-time image information, real-time key monitoring parameters and scoring function of the target coal transportation corridor, and to issue a fault warning according to the monitoring result.

[0108] In the specific implementation manner disclosed in the present application, the first acquisition 10 unit includes:

[0109] A second acquisition unit is used to acquire monitoring parameters of the coal conveying corridor, where the monitoring parameters are all parameters collected by sensors arranged in the coal conveying corridor;

[0110] The simplified element is used to simplify the monitoring parameters through the rough set reduction algorithm to obtain the monitoring parameters to be screened;

[0111] The first calculation unit is used to calculate the importance of the monitoring parameters to be screened to the fault adjustment information through a random forest algorithm, and determine the key monitoring parameters according to the importance.

[0112] In the specific implementation disclosed in this application, the unit 20 includes:

[0113] The second calculation unit is used to calculate the contribution of image features to the occurrence of faults based on a random forest algorithm, where the image features are features related to various faults on the image of the coal transportation corridor;

[0114] a third calculation unit, configured to calculate the determinism of the occurrence of a fault on the fault adjustment information based on a random forest algorithm;

[0115] The second construction unit is used to construct a recurrent neural network model based on the KERAS deep learning framework;

[0116] The setting unit is used to set the weight matrix of the recurrent neural network model based on the matrix calculation rule according to the contribution and the determinism to obtain the prediction model.

[0117] In the specific implementation manner disclosed in this application, the setting unit includes:

[0118] A third construction unit is used to construct a first matrix based on the contribution of each image feature to the occurrence of each fault and based on a matrix calculation rule;

[0119] A fourth construction unit, configured to construct a second matrix based on a matrix calculation rule based on the determinism of each fault occurrence on each fault adjustment information;

[0120] The second obtaining unit is used to use the first matrix and the second matrix as weight matrices of the hidden layer and the output layer of the recurrent neural network model respectively to obtain a prediction model.

[0121] In the specific embodiment disclosed in the present application, the first building block includes:

[0122] a third obtaining unit, configured to calculate a frequency score based on each key monitoring parameter and a preset range of each key monitoring parameter to obtain a plurality of first scores;

[0123] a fourth obtaining unit, configured to calculate an adjustment score based on the adjustment strength of each adjustment measure and the average adjustment strength of each adjustment measure, to obtain a plurality of second scores;

[0124] a fourth calculation unit, configured to calculate the product of the weight coefficient corresponding to each key monitoring parameter and the first score to obtain a plurality of first products;

[0125] A fifth calculation unit, used for calculating the product of the weight coefficient corresponding to each adjustment measure and the second score, to obtain a plurality of second products;

[0126] The sixth calculation unit is used to calculate the sum of the multiple first scores and the multiple second scores to obtain a scoring function.

[0127] In the specific implementation manner disclosed in the present application, the second obtaining unit includes:

[0128] A comparison unit, used to compare the key monitoring parameter with the corresponding preset range to obtain a comparison result;

[0129] A value taking unit, configured to take the first score value as a set threshold value when the comparison result indicates that the key monitoring parameter is within a preset range;

[0130] a seventh calculation unit, configured to calculate a difference between the key monitoring parameter and a lower limit of the preset range to obtain a first value when the comparison result indicates that the key monitoring parameter is not within the preset range;

[0131] an eighth calculation unit, configured to calculate a difference between an upper limit value and a lower limit value of a preset range to obtain a second value;

[0132] a ninth calculation unit, configured to calculate a difference between the first value and the second value to obtain a deviation range;

[0133] The tenth calculation unit is used to calculate an exponential function value with the natural constant as the base and the opposite number of the deviation range as the exponent as the first score.

[0134] In the specific implementation manner disclosed in this application, the third obtaining unit includes:

[0135] an eleventh calculating unit, configured to calculate a difference between an adjustment intensity of an adjustment measure and a corresponding average adjustment intensity to obtain a third value;

[0136] A twelfth calculation unit, used for calculating the ratio of the third value to the average adjustment intensity to obtain a fourth value;

[0137] The thirteenth calculation unit is used to calculate an exponential function value with the natural constant as the base and the opposite number of the fourth value as the exponent to obtain a second score.

[0138] In the specific implementation disclosed in the present application, the first obtaining unit 40 includes:

[0139] A fifth obtaining unit, used for calculating a real-time scoring function value based on the real-time image information and the real-time key monitoring parameters;

[0140] The sixth obtaining unit is used to map the scoring function value to a score range between 0 and 100 to obtain the safety quality score of the target coal transportation corridor;

[0141] The seventh obtaining unit is used to obtain a fault warning based on a preset safety quality scoring standard and a safety quality score.

[0142] It should be noted that, regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here. Embodiment 3:

[0143] Corresponding to the above method embodiment, this embodiment also provides a multimodal inspection device for a coal transportation corridor. The multimodal inspection device for a coal transportation corridor described below and the multimodal inspection method for a coal transportation corridor described above can be referred to each other.

[0144] Figure 4 8 is a block diagram of a multi-modal inspection device 800 for a coal transportation corridor according to an exemplary embodiment. Figure 4 As shown, the multi-modal inspection device 800 for coal transportation corridors may include: a processor 801 and a memory 802. The multi-modal inspection device 800 for coal transportation corridors may also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0145] The processor 801 is used to control the overall operation of the multimodal inspection device 800 for coal transportation corridors to complete all or part of the steps in the multimodal inspection method for coal transportation corridors described above. The memory 802 is used to store various types of data to support the operation of the multimodal inspection device 800 for coal transportation corridors, and these data may include, for example, instructions for any application or method for operating on the multimodal inspection device 800 for coal transportation corridors, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 802 or sent via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, and the above-mentioned other interface modules can be keyboards, mice, buttons, etc. These buttons can be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the multimodal inspection device 800 of the coal transportation corridor and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of them or several thereof, so the corresponding communication component 805 can include: Wi-Fi module, Bluetooth module, NFC module.

[0146] In an exemplary embodiment, the coal transportation corridor multimodal inspection device 800 can be implemented by one or more application specific integrated circuits (Application Specific Integrated Circuit, referred to as ASIC), digital signal processors (Digital Signal Processor, referred to as DSP), digital signal processing devices (Digital Signal Processing Device, referred to as DSPD), programmable logic devices (Programmable Logic Device, referred to as PLD), field programmable gate arrays (Field Programmable Gate Array, referred to as FPGA), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned coal transportation corridor multimodal inspection method.

[0147] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned multi-modal inspection method of the coal transportation corridor are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 802 including program instructions, and the above-mentioned program instructions can be executed by the processor 801 of the multi-modal inspection device 800 of the coal transportation corridor to complete the above-mentioned multi-modal inspection method of the coal transportation corridor. Embodiment 4:

[0148] Corresponding to the above method embodiment, a readable storage medium is also provided in this embodiment, and the readable storage medium described below and the multimodal inspection method of the coal transportation corridor described above can be referenced to each other.

[0149] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the multimodal inspection method for the coal transport corridor of the above-mentioned method embodiment.

[0150] The readable storage medium may specifically be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or other readable storage medium that can store program codes.

[0151] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0152] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A multi-modal inspection method for coal transportation corridors, characterized in that: include: Acquire first information and second information, wherein the first information is image information of the coal transportation corridor and corresponding fault adjustment information, and the second information is key monitoring parameters affecting the fault adjustment information; Taking the image information in the first information as input and the fault adjustment information as output, training and testing a preset prediction model to obtain a fault adjustment prediction model, wherein the preset prediction model is a recurrent neural network model based on KERAS; constructing a scoring function based on the fault adjustment prediction model and the second information; Based on the real-time image information of the target coal transportation corridor, the real-time key monitoring parameters and the scoring function, a monitoring result of the target coal transportation corridor is obtained, and a fault early warning is performed according to the monitoring result; Wherein, a scoring function is constructed based on the fault adjustment prediction model and the second information, and the fault adjustment information includes the adjustment strength of each adjustment measure, including: Calculating a frequency score based on each key monitoring parameter and a preset range of each key monitoring parameter to obtain a plurality of first scores; Calculate the adjustment score based on the adjustment strength of each adjustment measure and the average adjustment strength of each adjustment measure to obtain a plurality of second scores; Calculating the product of the weight coefficient corresponding to each key monitoring parameter and the first score to obtain multiple first products; Calculating the product of the weight coefficient corresponding to each adjustment measure and the second score to obtain multiple second products; The sum of the plurality of first scores and the plurality of second scores is calculated to obtain the score function.

2. The multi-modal inspection method for coal transportation corridor according to claim 1 is characterized in that , obtain the second information, including: Acquire monitoring parameters of the coal conveying corridor, wherein the monitoring parameters are all parameters collected by sensors arranged in the coal conveying corridor; Simplifying the monitoring parameters by using a rough set simplification algorithm to obtain monitoring parameters to be screened; The importance of the monitoring parameters to be screened to the fault adjustment information is calculated by a random forest algorithm, and the key monitoring parameters are determined according to the importance.

3. The multi-modal inspection method for coal transportation corridor according to claim 1 is characterized in that ,Calculate the frequency score based on each key monitoring parameter and the preset range of each key monitoring parameter, and obtain multiple ,first scores, including: Comparing the key monitoring parameter with the corresponding preset range to obtain a comparison result; When the comparison result indicates that the key monitoring parameter is within the preset range, the first score is set to a set threshold; When the comparison result indicates that the key monitoring parameter is not within the preset range, calculating the difference between the key monitoring parameter and the lower limit value of the preset range to obtain a first value; Calculate the difference between the upper limit value and the lower limit value of the preset range to obtain a second value; Calculating the difference between the first value and the second value to obtain a deviation range; An exponential function value having a natural constant as a base and an inverse number of the deviation range as an exponent is calculated as the first score.

4. The multi-modal inspection method for coal transportation corridor according to claim 1 is characterized in that , based on the regulation strength of each regulation measure and the average regulation strength of each regulation measure, the regulation score is calculated to obtain multiple second scores, including: Calculating the difference between the regulation intensity of the regulation measure and the corresponding average regulation intensity to obtain a third value; Calculating a ratio of the third value to the average adjustment intensity to obtain a fourth value; An exponential function value with a natural constant as a base and a negative number of the fourth value as an exponent is calculated to obtain the second score.

5. The multi-modal inspection method for coal transportation corridor according to claim 1 is characterized in that Based on the real-time image information of the target coal transportation corridor, the real-time key monitoring parameters and the scoring function, the monitoring result of the target coal transportation corridor is obtained, and a fault warning is performed according to the monitoring result, including: Calculate a real-time scoring function value based on the real-time image information and the real-time key monitoring parameters; Mapping the scoring function value to a score range between 0 and 100 to obtain a safety quality score for the target coal transportation corridor; The fault warning is obtained based on a preset safety quality scoring standard and the safety quality score.

6. The multi-modal inspection method for coal transportation corridor according to claim 4 is characterized in that ,The second scoring calculation formula is: ； in, For the A second score for each moderator; For the the intensity of each adjustment measure; For the The average moderating strength of the moderating measures.

7. The multi-modal inspection device for coal transportation corridor is characterized by: include: A first acquisition unit is used to acquire first information and second information, wherein the first information is image information of the coal transportation corridor and corresponding fault adjustment information, and the second information is a key monitoring parameter that affects the fault adjustment information; As a unit, used to take the image information in the first information as input and the fault adjustment information as output, train and test a preset prediction model to obtain a fault adjustment prediction model, wherein the preset prediction model is a recurrent neural network model based on KERAS; A first construction unit, configured to construct a scoring function based on the fault adjustment prediction model and the second information; A first obtaining unit is used to obtain a monitoring result of the target coal transportation corridor based on the real-time image information of the target coal transportation corridor, the real-time key monitoring parameters and the scoring function, and to perform a fault early warning according to the monitoring result; Wherein, the first building block comprises: a third obtaining unit, configured to calculate a frequency score based on each key monitoring parameter and a preset range of each key monitoring parameter to obtain a plurality of first scores; a fourth obtaining unit, configured to calculate an adjustment score based on the adjustment strength of each adjustment measure and the average adjustment strength of each adjustment measure, to obtain a plurality of second scores; a fourth calculation unit, configured to calculate the product of the weight coefficient corresponding to each key monitoring parameter and the first score to obtain a plurality of first products; A fifth calculation unit, used for calculating the product of the weight coefficient corresponding to each adjustment measure and the second score, to obtain a plurality of second products; The sixth calculation unit is used to calculate the sum of the multiple first scores and the multiple second scores to obtain a scoring function.

8. Multi-modal inspection equipment for coal transportation corridors, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of the multimodal inspection method for coal transportation corridors as described in any one of claims 1 to 6 when executing the computer program.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the multimodal inspection method for coal transportation corridors as described in any one of claims 1 to 6.

Images