Method and device for monitoring state of mine transport vehicle

Abstract

The invention relates to the technical field of vehicle state monitoring, in particular to a mine transport vehicle state monitoring method and device. The method comprises the following steps: acquiring video stream data and operation parameter data of a current transport vehicle in a preset time period; wherein the video stream data comprises image data of the current transport vehicle at a plurality of different moments, and the operation parameter data comprises speed, braking distance, brake pedal stroke and tire pressure, then determining comprehensive time sequence characteristics of the current transport vehicle according to the video stream data, determining motion characteristics of the current transport vehicle according to the operation parameter data, and determining the motion characteristics of the current transport vehicle according to the motion characteristics. Performing feature fusion on the comprehensive time sequence feature and the motion feature according to a trust contribution degree proportion to obtain a fused feature; and inputting the fusion features into a preset neural network prediction model to obtain the running state of the current transport vehicle, and through the configuration mode, the running state of the mine transport vehicle can be accurately monitored.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle status monitoring, and in particular to a method and device for monitoring the status of a mine transport vehicle. Background Art

[0002] In mine production, transport vehicles are key equipment to ensure the transportation of materials and personnel. However, the mine environment is complex, with flammable and explosive substances such as gas and dust, and the space is narrow and the light is dim. Transport vehicles face many safety hazards during operation. Traditional methods for monitoring the status of mine transport vehicles have many shortcomings, such as low monitoring accuracy, inability to monitor in real time, and inability to detect potential faults in a timely manner. This not only affects the production efficiency of the mine, but also poses a threat to the safety of personnel and equipment.

[0003] Based on this, there is an urgent need for a mine transport vehicle status monitoring method and device to solve the technical problem of how to accurately monitor the operating status of mine transport vehicles. Summary of the invention

[0004] In order to solve the technical problem of how to accurately monitor the operating status of a mine transport vehicle, an embodiment of the present invention provides a method and device for monitoring the status of a mine transport vehicle.

[0005] In a first aspect, an embodiment of the present invention provides a method for monitoring the status of a mine transport vehicle, the method being applied to a cloud, and the method comprising:

[0006] Acquire video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different times, and the operating parameter data includes speed, braking distance, brake pedal travel, and tire pressure;

[0007] Determine the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determine the motion characteristics of the current transport vehicle based on the operating parameter data;

[0008] The comprehensive time series feature and the motion feature are subjected to feature fusion according to a trust contribution ratio to obtain a fusion feature; wherein the trust contribution ratio is determined according to a weather influence factor and a route influence factor;

[0009] The fusion features are input into a preset neural network prediction model to obtain the operating status of the current transport vehicle.

[0010] In a second aspect, an embodiment of the present invention further provides a mine transport vehicle state monitoring and measuring device, comprising:

[0011] An acquisition module, used to acquire video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different times, and the operating parameter data includes speed, braking distance, brake pedal travel, and tire pressure;

[0012] A first data processing module, used to determine the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determine the motion characteristics of the current transport vehicle based on the operating parameter data;

[0013] A second data processing module is used to perform feature fusion on the comprehensive time series feature and the motion feature according to a trust contribution ratio to obtain a fusion feature; wherein the trust contribution ratio is determined according to a weather influence factor and a route influence factor;

[0014] The third data processing module is used to input the fusion features into a preset neural network prediction model to obtain the current operating status of the transport vehicle.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of the present invention is implemented.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method described in any embodiment of the present invention.

[0017] The embodiment of the present invention provides a method and device for monitoring the status of a mine transport vehicle. The cloud can collect video stream data of the current transport vehicle based on edge devices (such as smart street lights). The cloud and the transport vehicle are connected in communication. The transport vehicle can upload operating parameter data to the cloud in real time. The cloud determines the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determines the motion characteristics of the current transport vehicle based on the operating parameter data. The comprehensive time series characteristics and motion characteristics are fused according to the trust contribution ratio to obtain fused characteristics. The fused characteristics are input into a preset neural network prediction model to obtain the operating status of the current transport vehicle. When bad weather occurs, such as high concentrations of dust, strong humidity, etc., it will significantly interfere with the edge sensing device's collection of video stream data, resulting in a decrease in the signal-to-noise ratio of the data and an increase in image blur, which in turn leads to a decrease in data accuracy. Because the comprehensive time series features and motion features are fused according to the trust contribution ratio, and the trust contribution ratio is determined according to the weather influencing factors and the route influencing factors, the present invention can automatically reduce the weight of the comprehensive time series features in the fusion features by controlling the weather influencing factors, so as to effectively avoid the interference to the final detection results caused by the poor quality of video data, thereby improving the detection accuracy. When the transport vehicle is traveling in special sections such as mine tunnels, due to the complex electromagnetic environment, signal shielding effect and unique spatial topological structure in the tunnel, the stability of the communication link between the cloud server and the transport vehicle will be severely challenged, and data transmission delays, packet loss and bit errors will occur, which will seriously affect the accuracy and integrity of the motion feature data. In view of this situation, the present invention can automatically reduce the weight of the motion features in the fusion features by controlling the route influencing factors to ensure the reliability of the final detection results. In summary, the present invention can accurately monitor the operating status of mine transport vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is a flow chart of a method for monitoring the status of a mine transport vehicle provided by an embodiment of the present invention;

[0020] Figure 2 is a hardware architecture diagram of an electronic device provided by an embodiment of the present invention;

[0021] Figure 3 It is a structural diagram of a mine transport vehicle status monitoring device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0023] Please refer to Figure 1 The embodiment of the present invention provides a method for monitoring the state of a mine transport vehicle, the method comprising:

[0024] Step 100: Acquire video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different times, and the operating parameter data includes speed, braking distance, brake pedal travel, and tire pressure;

[0025] Step 102: Determine the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determine the motion characteristics of the current transport vehicle based on the operation parameter data;

[0026] Step 104: Fusing the comprehensive time series features and the motion features according to the trust contribution ratio to obtain a fusion feature; wherein the trust contribution ratio is determined according to the weather influence factor and the route influence factor;

[0027] Step 106: Input the fusion features into a preset neural network prediction model to obtain the current operating status of the transport vehicle.

[0028] In this embodiment, the cloud can collect video stream data of the current transport vehicle according to the edge device (such as smart street lamp), the cloud and the transport vehicle are connected in communication, and the transport vehicle can upload the operating parameter data to the cloud in real time. The cloud determines the comprehensive time series characteristics of the current transport vehicle according to the video stream data, and determines the motion characteristics of the current transport vehicle according to the operating parameter data. The comprehensive time series characteristics and motion characteristics are fused according to the trust contribution ratio to obtain the fusion characteristics, and the fusion characteristics are input into the preset neural network prediction model to obtain the operating status of the current transport vehicle. When bad weather occurs, such as high concentration of dust, strong humidity, etc., it will significantly interfere with the edge perception device to collect video stream data, resulting in a decrease in the signal-to-noise ratio of the data, an increase in image blur, and a decrease in data accuracy. Because the comprehensive time series characteristics and motion characteristics are fused according to the trust contribution ratio, and the trust contribution ratio is determined according to the weather influence factor and the route influence factor, the present invention can automatically reduce the weight of the comprehensive time series characteristics in the fusion characteristics by controlling the weather influence factor, so as to effectively avoid the interference caused by the poor quality of the video data on the final detection result, thereby improving the detection accuracy. When transport vehicles travel on special sections such as mine tunnels, due to the complex electromagnetic environment, signal shielding effect and unique spatial topology in the tunnel, the stability of the communication link between the cloud server and the transport vehicle will be severely challenged, and data transmission delays, packet loss and bit errors will occur, which will seriously affect the accuracy and integrity of the motion feature data. In view of this situation, the present invention can automatically reduce the weight of the motion feature in the fusion feature by controlling the route influencing factor, thereby ensuring the reliability of the final detection result. In summary, the present invention can accurately monitor the operating status of mine transport vehicles.

[0029] In one embodiment of the present invention, based on the video stream data, the comprehensive time series characteristics of the current transport vehicle are determined, including:

[0030] Extract features from the image data of the current transport vehicle at multiple different times to obtain a first feature and a second feature of the current transport vehicle; wherein the first feature is used to characterize the lateral feature of the current transport vehicle, and the second feature is used to characterize the longitudinal feature of the current transport vehicle;

[0031] The first feature and the second feature are superimposed to obtain a comprehensive time series feature.

[0032] In this embodiment, first, the image data of the current transport vehicle at multiple different times contained in the video stream data are processed using image feature extraction technology. This process aims to mine feature information that is critical to describing the vehicle state from these image data, and finally obtain the first feature and the second feature of the current transport vehicle. The first feature: This feature is mainly used to characterize the lateral features of the current transport vehicle. The lateral features cover various attributes and state information of the vehicle in the horizontal direction, such as the width of the vehicle, the change in the lateral profile, the lateral displacement, the inclination angle of the vehicle body on the horizontal plane, etc. These features can reflect the driving posture of the vehicle from the side, whether there is a lateral offset or swing, etc., and are used to judge the stability and motion state of the vehicle in the lateral dimension. The second feature: Its function is to characterize the longitudinal features of the current transport vehicle. The longitudinal features mainly involve the relevant attributes and states of the vehicle in the front and rear directions, including the length of the vehicle, the dynamic changes of the longitudinal profile, the motion state of the front and rear wheels, and the performance of the acceleration and deceleration of the vehicle in the longitudinal direction. This information helps to analyze the changes in the vehicle's driving speed, braking performance, and driving stability in the longitudinal dimension. After successfully extracting the first feature and the second feature, the present invention performs a feature superposition operation on the two features to obtain a comprehensive time series feature. Feature superposition is a method of integrating features of different dimensions or types. Its purpose is to fuse the lateral and longitudinal features of the vehicle to form a more comprehensive feature representation that better reflects the overall motion state of the vehicle.

[0033] In one embodiment of the present invention, the trust contribution ratio is determined by the following formula:

[0034]

[0035] In the formula, is the trust contribution ratio, x is the weather influencing factor, y is the route influencing factor, w1 is the first weight coefficient, w2 is the second weight coefficient, λ is the total length of the route traveled by the current transport vehicle within the preset time period, t1 is the starting time point of the preset time period, and t2 is the ending time point of the preset time period.

[0036] In this embodiment, technicians in this field can customize the weather influence factor, route influence factor, first weight coefficient, second weight coefficient, the total length of the route traveled by the current transport vehicle within the preset time period, the starting time point of the preset time period, and the ending time point of the preset time period according to actual usage.

[0037] In one embodiment of the present invention, after the fusion features are input into a preset neural network prediction model to obtain the current operating status of the transport vehicle, the following steps are included:

[0038] Obtain the operating status of the front transport vehicle that is closest to the current transport vehicle;

[0039] Selectively enter the risk control process based on the operating status of the preceding transport vehicle that is closest to the current transport vehicle and the operating status of the current transport vehicle;

[0040] When entering the risk control process, the risk coefficient is determined according to the risk dynamic rating equation group;

[0041] Based on the risk factor, the final operating status of the transport vehicle is determined.

[0042] In this embodiment, the operating status of the front transport vehicle which is closest to the current transport vehicle is obtained.

[0043] Based on the acquired operating status of the current transport vehicle and the nearest transport vehicle in front, it is selectively decided whether to enter the risk control process. When the present invention determines to enter the risk control process, the risk coefficient will be determined according to the risk dynamic rating equation group. This coefficient can intuitively reflect the degree of risk faced by the current transportation situation. After obtaining the risk coefficient, the operating status of the transport vehicle will be re-evaluated based on this to determine the final operating status of the transport vehicle. This process will be classified according to the size of the risk coefficient. For example, when the risk coefficient is low, the vehicle is still in a normal operating state, but the driver needs to remain vigilant; when the risk coefficient reaches a certain threshold, the present invention will determine that the vehicle is in a warning state, and may take corresponding measures, such as issuing an alarm to remind the driver, automatically adjusting the speed, etc.; if the risk coefficient is extremely high, the present invention may determine that the vehicle is in a dangerous state, and trigger emergency braking and other safety measures to ensure the safety of the vehicle and personnel. Therefore, the present invention can fully consider the operating conditions and potential risks of surrounding vehicles, dynamically adjust the evaluation of the operating status of the transport vehicle, thereby achieving more accurate and reliable risk control and safety assurance.

[0044] In one embodiment of the present invention, the operating state includes a normal state and an abnormal state;

[0045] Based on the operating status of the preceding transport vehicle closest to the current transport vehicle and the operating status of the current transport vehicle, the risk control process is selectively entered, including:

[0046] Determine whether the current operating state of the transport vehicle is an abnormal state;

[0047] If yes, then enter the risk control process;

[0048] If not, when the operating status of the front transport vehicle closest to the current transport vehicle is abnormal, the risk control process is entered.

[0049] In this embodiment, the current vehicle is abnormal: when the running state of the current transport vehicle is judged to be abnormal, the risk control process is directly entered. This is because when the current vehicle itself has problems, it is very likely to cause a safety accident, and immediate measures need to be taken to deal with it, such as deceleration, parking inspection, etc. The current vehicle is normal but the vehicle in front is abnormal: if the running state of the current transport vehicle is normal, further attention will be paid to the running state of the transport vehicle in front that is closest to the current transport vehicle. When the vehicle in front is detected to be in an abnormal state, the risk control process is also entered. This is to take into account that the abnormality of the vehicle in front may have an impact on the current vehicle, such as the sudden stop or failure of the vehicle in front, which may cause the current vehicle to collide before it can react. In this way, the present invention can predict potential dangers in advance, take preventive measures in time, and improve the safety of the entire transportation process. When the current vehicle and the vehicle in front are operating normally, there is no need to start the risk control process, which ensures transportation efficiency; and when an abnormal situation occurs, the risk control process is entered in time, which can effectively reduce the probability of accidents and ensure the safety of personnel and equipment.

[0050] In one embodiment of the present invention, the risk dynamic rating equation group is constructed by the following formula:

[0051]

[0052] Where N is the risk factor, t1 is the starting time point of the preset time period, t2 is the ending time point of the preset time period, Δt is the time interval of data sampling, and w(t i ) is the time weight coefficient, s th is the preset safety distance, m g (t i ) is the influence coefficient of the vehicle's own state on distance judgment, s(t i ) at t i The distance between the current transport vehicle and the nearest transport vehicle ahead at the moment. is the correction coefficient under the same road condition, u(x) is the unit step function, a is the acceleration of the current transport vehicle, and a max is the maximum acceleration of the current transport vehicle, v is the speed of the current transport vehicle, v0 is the expected speed of the current transport vehicle, is the acceleration index, s * (v, Δv) is the expected safe following distance, s0 is the minimum safe distance, T is the driver's preset reaction time, Δv is the speed difference between the vehicle and the preceding vehicle, b is the maximum deceleration, s min Vehicles maintain a minimum safe distance.

[0053] In this embodiment, w(t i ) is the time weight coefficient, which means that at t iThe weight corresponding to the moment represents the difference in the importance of the risk factor in different time periods. u(x) is a unit step function, which determines the output value according to the size of the input value x. m g (t i ) is the influence coefficient of the vehicle's own state on distance judgment, m g (t i ) value will change according to the different states of the vehicle and is used to correct the preset distance. For example, when the vehicle is in a braking state, in order to ensure safety, the preset distance needs to be appropriately reduced. At this time, m g (t i ) will be adjusted accordingly. is the correction coefficient under the same road conditions, The value of will change according to different road conditions and is used to correct the actual distance. For example, in congested road conditions, the distance between vehicles is generally small. The value of can be appropriately increased to more accurately reflect the actual situation. The risk coefficient can be accurately solved through the risk dynamic rating equation group, thereby ensuring the accuracy of monitoring the operating status of mine transportation vehicles.

[0054] In one embodiment of the present invention, after determining the final operating state of the transport vehicle based on the risk factor, the method further includes:

[0055] Determine a combination of risk warning information according to the risk coefficient, and send the combination of risk warning information to the vehicle end;

[0056] Among them, the risk warning information includes voice prompts, single driver's seat vibration, multiple driver's seat vibrations, and single passenger seat vibration.

[0057] In this embodiment, after determining the final operating state of the transport vehicle, the present invention will further generate a combination of risk warning information based on the risk coefficient, and send the combination to the vehicle end. The risk warning information covers voice prompts, different forms of vibration of the driver and co-driver seats, etc. The present invention will select appropriate prompt methods for combination according to the size of the risk coefficient to attract the driver's attention. After determining the combination of risk warning information, the present invention will send the information to the vehicle end through the communication link between the vehicle and the cloud. After the vehicle end receives the information, the corresponding equipment (such as the voice broadcast of the present invention and the seat vibration device) will work in combination.

[0058] like Figure 2 , Figure 3 As shown, an embodiment of the present invention provides a device for monitoring and measuring the state of a mine transport vehicle. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From the hardware level, Figure 2As shown, it is a hardware architecture diagram of an electronic device in which a mine transport vehicle status monitoring and measuring device provided by an embodiment of the present invention is located. Figure 2 In addition to the processor, memory, network interface, and non-volatile memory shown, the electronic device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 3 As shown, as a device in a logical sense, the CPU of the electronic device in which it is located reads the corresponding computer program in the non-volatile memory into the internal memory and runs it.

[0059] like Figure 3 As shown, this embodiment provides a mine transport vehicle status monitoring and measuring device, the device comprising:

[0060] The acquisition module 300 is used to acquire the video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different times, and the operating parameter data includes speed, braking distance, brake pedal travel, and tire pressure;

[0061] A first data processing module 302 is used to determine the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determine the motion characteristics of the current transport vehicle based on the operating parameter data;

[0062] The second data processing module 304 is used to fuse the comprehensive time series features and the motion features according to the trust contribution ratio to obtain a fusion feature; wherein the trust contribution ratio is determined according to the weather influence factor and the route influence factor;

[0063] The third data processing module 306 is used to input the fusion features into a preset neural network prediction model to obtain the current operating status of the transport vehicle.

[0064] In one embodiment of the present invention, the trust contribution ratio is determined by the following formula:

[0065]

[0066] In the formula, is the trust contribution ratio, x is the weather influencing factor, y is the route influencing factor, w1 is the first weight coefficient, w2 is the second weight coefficient, λ is the total length of the route traveled by the current transport vehicle within the preset time period, t1 is the starting time point of the preset time period, and t2 is the ending time point of the preset time period.

[0067] In one embodiment of the present invention, based on the video stream data, determining the comprehensive time series characteristics of the current transport vehicle includes:

[0068] Extract features from the image data of the current transport vehicle at the multiple different moments to obtain a first feature and a second feature of the current transport vehicle; wherein the first feature is used to characterize a lateral feature of the current transport vehicle, and the second feature is used to characterize a longitudinal feature of the current transport vehicle;

[0069] The first feature and the second feature are superimposed to obtain the comprehensive time series feature.

[0070] In one embodiment of the present invention, after inputting the fusion features into a preset neural network prediction model to obtain the current operating status of the transport vehicle, the method further includes:

[0071] Obtain the operating status of the front transport vehicle that is closest to the current transport vehicle;

[0072] Selectively enter the risk control process based on the operating status of the preceding transport vehicle that is closest to the current transport vehicle and the operating status of the current transport vehicle;

[0073] When entering the risk control process, the risk coefficient is determined according to the risk dynamic rating equation group;

[0074] Based on the risk factor, the final operating status of the transport vehicle is determined.

[0075] In one embodiment of the present invention, the operating state includes a normal state and an abnormal state;

[0076] The selectively entering the risk control process based on the operating status of the front transport vehicle closest to the current transport vehicle and the operating status of the current transport vehicle includes:

[0077] Determining whether the current operating state of the transport vehicle is an abnormal state;

[0078] If yes, then enter the risk control process;

[0079] If not, when the operating state of the front transport vehicle that is closest to the current transport vehicle is abnormal, the risk control process is entered.

[0080] In one embodiment of the present invention, the risk dynamic rating equation group is constructed by the following formula:

[0081]

[0082]

[0083] Where N is the risk factor, t1 is the starting time point of the preset time period, t2 is the ending time point of the preset time period, Δt is the time interval of data sampling, w(t i ) is the time weight coefficient, s th is the preset safety distance, m g (t i ) is the influence coefficient of the vehicle's own state on distance judgment, s(t i ) at t i The distance between the current transport vehicle and the nearest transport vehicle ahead at the moment. is the correction coefficient under the same road condition, u(x) is the unit step function, a is the acceleration of the current transport vehicle, and a max is the maximum acceleration of the current transport vehicle, v is the speed of the current transport vehicle, v0 is the expected speed of the current transport vehicle, is the acceleration index, s * (v, Δv) is the expected safe following distance, s0 is the minimum safe distance, T is the driver's preset reaction time, Δv is the speed difference between the vehicle and the preceding vehicle, b is the maximum deceleration, s min Vehicles maintain a minimum safe distance.

[0084] In one embodiment of the present invention, after determining the final operating state of the transport vehicle based on the risk coefficient, the method further includes:

[0085] Determine a combination of risk warning information according to the risk coefficient, and send the combination of risk warning information to the vehicle end;

[0086] Among them, the risk warning information includes voice prompts, single driver's seat vibration, multiple driver's seat vibrations, and single passenger seat vibration.

[0087] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on a device for monitoring and measuring the state of a mine transport vehicle. In other embodiments of the present invention, a device for monitoring and measuring the state of a mine transport vehicle may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0088] The information interaction, execution process and other contents between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For the specific contents, please refer to the description in the embodiment of the method of the present invention, and no further description is given here.

[0089] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a method for monitoring the status of a mine transport vehicle in any embodiment of the present invention is implemented.

[0090] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes a method for monitoring the status of a mine transport vehicle in any embodiment of the present invention.

[0091] Specifically, a system or device equipped with a storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program codes stored in the storage medium.

[0092] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0093] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0094] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0095] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0096] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0097] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring the status of a mine transport vehicle, characterized in that: The method is applied to the cloud, and the method includes: Acquire video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different times, and the operating parameter data includes speed, braking distance, brake pedal travel, and tire pressure; Determine the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determine the motion characteristics of the current transport vehicle based on the operating parameter data; The comprehensive time series feature and the motion feature are subjected to feature fusion according to a trust contribution ratio to obtain a fusion feature; wherein the trust contribution ratio is determined according to a weather influence factor and a route influence factor; The fusion features are input into a preset neural network prediction model to obtain the current operating status of the transport vehicle.

2. The method according to claim 1, characterized in that The trust contribution ratio is determined by the following formula: In the formula, is the trust contribution ratio, x is the weather influencing factor, y is the route influencing factor, w1 is the first weight coefficient, w2 is the second weight coefficient, λ is the total length of the route traveled by the current transport vehicle within the preset time period, t1 is the starting time point of the preset time period, and t2 is the ending time point of the preset time period.

3. The method according to claim 2, characterized in that Based on the video stream data, the comprehensive time sequence characteristics of the current transport vehicle are determined, including: Extract features from the image data of the current transport vehicle at the multiple different moments to obtain a first feature and a second feature of the current transport vehicle; wherein the first feature is used to characterize a lateral feature of the current transport vehicle, and the second feature is used to characterize a longitudinal feature of the current transport vehicle; The first feature and the second feature are superimposed to obtain the comprehensive time series feature.

4. The method according to claim 3, characterized in that After the fusion feature is input into a preset neural network prediction model to obtain the current operation status of the transport vehicle, the method includes: Obtain the operating status of the front transport vehicle that is closest to the current transport vehicle; Selectively enter the risk control process based on the operating status of the preceding transport vehicle that is closest to the current transport vehicle and the operating status of the current transport vehicle; When entering the risk control process, the risk coefficient is determined according to the risk dynamic rating equation group; Based on the risk factor, the final operating status of the transport vehicle is determined.

5. The method according to claim 4, characterized in that The operating state includes a normal state and an abnormal state; The selectively entering the risk control process based on the operating status of the front transport vehicle closest to the current transport vehicle and the operating status of the current transport vehicle includes: Determining whether the current operating state of the transport vehicle is an abnormal state; If yes, then enter the risk control process; If not, when the operating state of the front transport vehicle that is closest to the current transport vehicle is abnormal, the risk control process is entered.

6. The method according to claim 5, characterized in that The risk dynamic rating equation group is constructed by the following formula: Where N is the risk factor, t1 is the starting time point of the preset time period, t2 is the ending time point of the preset time period, Δt is the time interval of data sampling, w(t i ) is the time weight coefficient, s th is the preset safety distance, m g (t i ) is the influence coefficient of the vehicle's own state on distance judgment, s(t i ) at t i The distance between the current transport vehicle and the nearest transport vehicle ahead at the moment, k f(ti) is the correction coefficient under the same road condition, u(x) is the unit step function, a is the acceleration of the current transport vehicle, and a max is the maximum acceleration of the current transport vehicle, v is the speed of the current transport vehicle, v0 is the expected speed of the current transport vehicle, is the acceleration index, s * (v, Δv) is the expected safe following distance, s0 is the minimum safe distance, T is the driver's preset reaction time, Δv is the speed difference between the vehicle and the preceding vehicle, b is the maximum deceleration, s min Vehicles maintain a minimum safe distance.

7. The method according to claim 4, characterized in that After determining the final operation state of the transport vehicle based on the risk factor, the method further includes: Determine a combination of risk warning information according to the risk coefficient, and send the combination of risk warning information to the vehicle end; Among them, the risk warning information includes voice prompts, single driver's seat vibration, multiple driver's seat vibrations, and single passenger seat vibration.

8. A mine transport vehicle status monitoring device, characterized in that: include: An acquisition module, used to acquire video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different times, and the operating parameter data includes speed, braking distance, brake pedal travel, and tire pressure; A first data processing module, used to determine the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determine the motion characteristics of the current transport vehicle based on the operating parameter data; A second data processing module is used to perform feature fusion on the comprehensive time series feature and the motion feature according to a trust contribution ratio to obtain a fusion feature; wherein the trust contribution ratio is determined according to a weather influencing factor and a route influencing factor; The third data processing module is used to input the fusion features into a preset neural network prediction model to obtain the current operating status of the transport vehicle.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images