Mine transport vehicle status monitoring method and device

Abstract

The present invention relates to the field of vehicle status monitoring technology, and in particular to a method and device for monitoring the status of a mine transport vehicle. The present invention obtains video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different times, and the operating parameter data includes speed, braking distance, brake pedal stroke, and tire pressure, and then determines the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determines the motion characteristics of the current transport vehicle based on the operating parameter data, and fuses the comprehensive time series characteristics and motion characteristics according to the trust contribution ratio to obtain fused characteristics; and inputs the fused characteristics into a preset neural network prediction model to obtain the operating status of the current transport vehicle. Through the above configuration, the present invention can accurately monitor the operating status of mine transport vehicles.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle status monitoring, and in particular to a method and device for monitoring the status of a mine transport vehicle. Background Art

[0002] In mine operations, transport vehicles are critical equipment for transporting materials and personnel. However, mine environments are complex, with the presence of flammable and explosive substances such as gas and dust, as well as cramped spaces and dim lighting. Transport vehicles face numerous safety hazards during operation. Traditional methods for monitoring the status of mine transport vehicles have numerous shortcomings, including low accuracy, inability to monitor in real time, and inability to detect potential faults. This not only impacts mine production efficiency but also poses a threat to the safety of personnel and equipment.

[0003] Based on this, there is an urgent need for a mine transport vehicle status monitoring method and device to solve the technical problem of how to accurately monitor the operating status of mine transport vehicles. Summary of the Invention

[0004] In order to solve the technical problem of how to accurately monitor the operating status of a mine transport vehicle, an embodiment of the present invention provides a method and device for monitoring the status of a mine transport vehicle.

[0005] In a first aspect, an embodiment of the present invention provides a method for monitoring the status of a mine transport vehicle, the method being applied to a cloud and comprising:

[0006] Obtaining video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different times, and the operating parameter data includes speed, braking distance, brake pedal travel, and tire pressure;

[0007] Determining the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determining the motion characteristics of the current transport vehicle based on the operating parameter data;

[0008] Fusing the comprehensive time series features and the motion features according to a trust contribution ratio to obtain a fused feature; wherein the trust contribution ratio is determined based on a weather influencing factor and a route influencing factor;

[0009] The fusion features are input into a preset neural network prediction model to obtain the current operating status of the transport vehicle.

[0010] In a second aspect, an embodiment of the present invention further provides a mine transport vehicle status monitoring and measurement device, comprising:

[0011] an acquisition module, configured to acquire video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different moments, and the operating parameter data includes speed, braking distance, brake pedal travel, and tire pressure;

[0012] a first data processing module, configured to determine the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determine the motion characteristics of the current transport vehicle based on the operating parameter data;

[0013] a second data processing module, configured to perform feature fusion on the comprehensive time series features and the motion features according to a trust contribution ratio to obtain a fused feature; wherein the trust contribution ratio is determined based on a weather influencing factor and a route influencing factor;

[0014] The third data processing module is used to input the fusion features into a preset neural network prediction model to obtain the current operating status of the transport vehicle.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any embodiment of the present invention is implemented.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method described in any embodiment of the present invention.

[0017] The embodiment of the present invention provides a method and device for monitoring the status of mine transport vehicles. The cloud can collect video stream data of the current transport vehicle based on edge devices (such as smart street lights). The cloud and the transport vehicle are connected in communication, and the transport vehicle can upload operating parameter data to the cloud in real time. The cloud then determines the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determines the motion characteristics of the current transport vehicle based on the operating parameter data. The comprehensive time series characteristics and motion characteristics are fused according to the trust contribution ratio to obtain fused characteristics. The fused characteristics are input into a preset neural network prediction model to obtain the operating status of the current transport vehicle. When severe weather occurs, such as high concentrations of dust and strong humidity, it will significantly interfere with the edge sensing device's collection of video stream data, resulting in a decrease in the signal-to-noise ratio of the data and an increase in image blur, which in turn leads to a decrease in data accuracy. Because the comprehensive time series features and motion features are fused according to the trust contribution ratio, and the trust contribution ratio is determined based on the weather influencing factors and the route influencing factors, the present invention can automatically reduce the weight ratio of the comprehensive time series features in the fusion features by controlling the weather influencing factors, thereby effectively avoiding the interference on the final detection results caused by the poor quality of the video data, and thus improving the detection accuracy. When the transport vehicle is traveling on special sections such as mine tunnels, due to the complex electromagnetic environment, signal shielding effect and unique spatial topology in the tunnel, the stability of the communication link between the cloud server and the transport vehicle will be severely challenged, and data transmission delays, packet loss and bit errors will occur, which will seriously affect the accuracy and integrity of the motion feature data. In response to this situation, the present invention can automatically reduce the weight ratio of the motion feature in the fusion features by controlling the route influencing factors, thereby ensuring the reliability of the final detection results. In summary, the present invention can accurately monitor the operating status of mine transport vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a flow chart of a method for monitoring the status of a mine transport vehicle provided by an embodiment of the present invention;

[0020] Figure 2 This is a hardware architecture diagram of an electronic device provided by an embodiment of the present invention;

[0021] Figure 3 This is a structural diagram of a mine transport vehicle status monitoring device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0023] Please refer to Figure 1 , an embodiment of the present invention provides a method for monitoring the status of a mine transport vehicle, the method comprising:

[0024] Step 100: Acquire video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different moments, and the operating parameter data includes speed, braking distance, brake pedal travel, and tire pressure;

[0025] Step 102: Determine the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determine the motion characteristics of the current transport vehicle based on the operating parameter data;

[0026] Step 104: Fusing the comprehensive time series features and motion features according to the trust contribution ratio to obtain a fused feature; wherein the trust contribution ratio is determined based on weather influencing factors and route influencing factors;

[0027] Step 106: Input the fused features into a preset neural network prediction model to obtain the current operating status of the transport vehicle.

[0028] In this embodiment, the cloud can collect video stream data of the current transport vehicle based on edge devices (such as smart street lights). The cloud and the transport vehicle are connected in communication, and the transport vehicle can upload operating parameter data to the cloud in real time. The cloud then determines the comprehensive time series features of the current transport vehicle based on the video stream data, and determines the motion features of the current transport vehicle based on the operating parameter data. The comprehensive time series features and motion features are fused according to the trust contribution ratio to obtain a fused feature. The fused feature is input into a preset neural network prediction model to obtain the operating status of the current transport vehicle. When inclement weather occurs, such as high concentrations of dust and strong humidity, it will significantly interfere with the edge sensing device's collection of video stream data, resulting in a decrease in the data's signal-to-noise ratio and increased image blur, which in turn leads to a decrease in data accuracy. Because the comprehensive time series features and motion features are fused according to the trust contribution ratio, which is determined based on weather influencing factors and route influencing factors, the present invention can automatically reduce the weight of the comprehensive time series features in the fused feature by controlling the weather influencing factor, thereby effectively avoiding the interference caused by poor video data quality on the final detection result, thereby improving detection accuracy. When transport vehicles travel on special sections of roads such as mine tunnels, the stability of the communication link between the cloud server and the transport vehicle will be severely challenged due to the complex electromagnetic environment, signal shielding effect, and unique spatial topology in the tunnel. Data transmission delays, packet loss, and bit errors will occur, which will seriously affect the accuracy and integrity of the motion feature data. In response to this situation, the present invention can automatically reduce the weight of the motion feature in the fusion feature by controlling the route influencing factor, thereby ensuring the reliability of the final detection result. In summary, the present invention can accurately monitor the operating status of mine transport vehicles.

[0029] In one embodiment of the present invention, determining the comprehensive temporal characteristics of the current transport vehicle based on the video stream data includes:

[0030] Extracting features from the image data of the current transport vehicle at multiple different moments to obtain a first feature and a second feature of the current transport vehicle; wherein the first feature is used to characterize a lateral feature of the current transport vehicle, and the second feature is used to characterize a longitudinal feature of the current transport vehicle;

[0031] The first feature and the second feature are superimposed to obtain a comprehensive time series feature.

[0032] In this embodiment, image feature extraction techniques are first used to process the image data of the current transport vehicle at multiple different times contained in the video stream data. This process aims to extract key feature information from this image data that is crucial for describing the vehicle's state, ultimately obtaining the first and second features of the current transport vehicle. The first feature primarily characterizes the lateral characteristics of the current transport vehicle. These lateral features encompass various attributes and status information about the vehicle in the horizontal direction, such as its width, lateral profile changes, lateral displacement, and the vehicle's tilt angle in the horizontal plane. These features can reflect the vehicle's driving posture, including any lateral offset or sway, and are used to determine the vehicle's stability and motion in the lateral dimension. The second feature characterizes the longitudinal characteristics of the current transport vehicle. These longitudinal features primarily involve relevant attributes and states in the fore-aft direction, including the vehicle's length, dynamic changes in the longitudinal profile, the motion of the front and rear wheels, and the vehicle's acceleration and deceleration in the longitudinal direction. This information facilitates analysis of vehicle speed fluctuations, braking performance, and longitudinal driving stability. After successfully extracting the first and second features, the present invention performs a feature overlay operation on these two features to obtain a comprehensive time series feature. Feature superposition is a method of integrating features of different dimensions or types. Its purpose is to fuse the lateral and longitudinal features of the vehicle to form a more comprehensive feature representation that better reflects the overall motion state of the vehicle.

[0033] In one embodiment of the present invention, the trust contribution ratio is determined by the following formula:

[0034]

[0035] Where, is the trust contribution ratio, x is the weather influencing factor, y is the route influencing factor, w1 is the first weight coefficient, w2 is the second weight coefficient, λ is the total length of the route traveled by the current transport vehicle within the preset time period, t1 is the starting time point of the preset time period, and t2 is the ending time point of the preset time period.

[0036] In this embodiment, those skilled in the art can customize the weather influence factor, route influence factor, first weight coefficient, second weight coefficient, the total length of the route traveled by the current transport vehicle within the preset time period, the starting time point of the preset time period, and the ending time point of the preset time period according to actual usage.

[0037] In one embodiment of the present invention, after the fusion features are input into a preset neural network prediction model to obtain the current operating status of the transport vehicle, the following steps are included:

[0038] Get the running status of the transport vehicle closest to the current transport vehicle;

[0039] Selectively enter the risk control process based on the operating status of the preceding transport vehicle closest to the current transport vehicle and the operating status of the current transport vehicle;

[0040] When entering the risk control process, the risk coefficient is determined according to the risk dynamic rating equation group;

[0041] Based on the risk factor, the final operating status of the transport vehicle is determined.

[0042] In this embodiment, the operating status of the preceding transport vehicle that is closest to the current transport vehicle is obtained.

[0043] Based on the acquired operating status of the current transport vehicle and its nearest preceding transport vehicle, the system selectively determines whether to enter the risk control process. Once the risk control process is entered, the system determines a risk coefficient based on a dynamic risk rating equation. This coefficient intuitively reflects the degree of risk faced by the current transport situation. Once the risk coefficient is determined, the transport vehicle's operating status is reassessed to determine the final operating status. This process categorizes the process based on the magnitude of the risk coefficient. For example, when the risk coefficient is low, the vehicle remains in normal operation, but the driver must remain vigilant. When the risk coefficient reaches a certain threshold, the system determines that the vehicle is in a warning state and may take appropriate measures, such as sounding an alarm to alert the driver or automatically adjusting the vehicle's speed. If the risk coefficient is extremely high, the system may determine that the vehicle is in a dangerous state and trigger safety measures such as emergency braking to ensure the safety of the vehicle and personnel. Therefore, the system can fully consider the operating conditions and potential risks of surrounding vehicles and dynamically adjust the assessment of the transport vehicle's operating status, thereby achieving more accurate and reliable risk control and safety assurance.

[0044] In one embodiment of the present invention, the operating state includes a normal state and an abnormal state;

[0045] Based on the operating status of the nearest preceding transport vehicle and the current transport vehicle, the risk control process is selectively initiated, including:

[0046] Determine whether the current operating status of the transport vehicle is abnormal;

[0047] If yes, then enter the risk control process;

[0048] If not, when the operating status of the front transport vehicle closest to the current transport vehicle is abnormal, the risk control process is entered.

[0049] In this embodiment, if the current vehicle is abnormal: When the operating status of the current transport vehicle is determined to be abnormal, the risk control process is directly initiated. This is because problems with the current vehicle itself are highly likely to cause a safety incident, requiring immediate action such as deceleration or a stop for inspection. If the current vehicle is normal but the preceding vehicle is abnormal: If the current vehicle is normal, the system will further examine the operating status of the preceding vehicle closest to the current vehicle. If the preceding vehicle is detected to be abnormal, the risk control process is also initiated. This is because an abnormality in the preceding vehicle may affect the current vehicle. For example, a sudden stop or malfunction of the preceding vehicle could prevent the current vehicle from reacting and potentially causing a collision. In this way, the present invention can predict potential hazards in advance, implement preventive measures promptly, and improve the safety of the entire transport process. When both the current vehicle and the preceding vehicle are operating normally, there is no need to initiate the risk control process, ensuring transport efficiency. However, when an abnormality occurs, the risk control process is promptly initiated, effectively reducing the probability of accidents and ensuring the safety of personnel and equipment.

[0050] In one embodiment of the present invention, the risk dynamic rating equation group is constructed by the following formula:

[0051]

[0052] Where N is the risk factor, t1 is the starting time point of the preset time period, t2 is the ending time point of the preset time period, Δt is the time interval of data sampling, w(t i ) is the time weight coefficient, s th is the preset safety distance, m g (t i ) is the influence coefficient of the vehicle's own state on distance judgment, s(t i ) at t i The distance between the current transport vehicle and the nearest transport vehicle ahead at the moment, is the correction coefficient under the same road conditions, u(x) is the unit step function, a is the acceleration of the current transport vehicle, and a max is the maximum acceleration of the current transport vehicle, v is the speed of the current transport vehicle, v0 is the expected speed of the current transport vehicle, is the acceleration index, s * (v, Δv) is the desired safe following distance, s0 is the minimum safe distance, T is the driver's preset reaction time, Δv is the speed difference between the vehicle and the preceding vehicle, b is the maximum deceleration, s min Vehicles maintain a minimum safe distance.

[0053] In this embodiment, w(t i ) is the time weight coefficient, which means that at t iThe weight corresponding to the moment represents the difference in the importance of the risk coefficient in different time periods. u(x) is a unit step function, which determines the output value according to the size of the input value x. m g (t i ) is the influence coefficient of the vehicle's own state on distance judgment, m g (t i ) value will change according to the different states of the vehicle and is used to correct the preset distance. For example, when the vehicle is in the braking state, in order to ensure safety, it is necessary to appropriately reduce the preset distance. At this time, m g (t i ) will be adjusted accordingly. is the correction coefficient under the same road conditions, The value of will change according to different road conditions and is used to correct the actual distance. For example, in congested road conditions, the distance between vehicles is generally small. The value of can be appropriately increased to more accurately reflect the actual situation. The risk coefficient can be accurately solved through the risk dynamic rating equation group, thereby ensuring the accuracy of monitoring the operating status of mine transportation vehicles.

[0054] In one embodiment of the present invention, after determining the final operating state of the transport vehicle based on the risk factor, the method further includes:

[0055] Determine a combination of risk warning information based on the risk coefficient, and send the combination of risk warning information to the vehicle end;

[0056] Among them, risk warning information includes voice prompts, single driver's seat vibration, multiple driver's seat vibrations, and single passenger seat vibration.

[0057] In this embodiment, after determining the final operating status of the transport vehicle, the present invention will further generate a combination of risk warning information based on the risk coefficient, and send the combination to the vehicle end. The risk warning information covers voice prompts, different forms of vibration of the driver and co-driver seats, etc. The present invention will select appropriate prompt methods for combination according to the size of the risk coefficient to attract the driver's attention. After determining the combination of risk warning information, the present invention will send the information to the vehicle end through the communication link between the vehicle and the cloud. After the vehicle end receives the information, the corresponding equipment (such as the voice broadcast of the present invention and the seat vibration device) will work in combination.

[0058] like Figure 2 、 Figure 3 As shown, the embodiment of the present invention provides a device for monitoring and measuring the state of a mine transport vehicle. The device embodiment can be implemented by software, hardware, or a combination of software and hardware. From the hardware level, as Figure 2The figure shows a hardware architecture diagram of an electronic device where a mine transport vehicle status monitoring and measuring device is located, in addition to Figure 2 In addition to the processor, memory, network interface, and non-volatile memory shown, the electronic device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 3 As shown, as a device in a logical sense, it is formed by the CPU of the electronic device in which it is located reading the corresponding computer program in the non-volatile memory into the internal memory and running it.

[0059] like Figure 3 As shown, this embodiment provides a mine transport vehicle status monitoring and measurement device, the device comprising:

[0060] An acquisition module 300 is configured to acquire video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different moments, and the operating parameter data includes speed, braking distance, brake pedal travel, and tire pressure;

[0061] A first data processing module 302 is configured to determine the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determine the motion characteristics of the current transport vehicle based on the operating parameter data;

[0062] A second data processing module 304 is configured to perform feature fusion on the comprehensive time series features and the motion features according to a trust contribution ratio to obtain a fused feature; wherein the trust contribution ratio is determined based on a weather influencing factor and a route influencing factor;

[0063] The third data processing module 306 is used to input the fusion features into a preset neural network prediction model to obtain the current operating status of the transport vehicle.

[0064] In one embodiment of the present invention, the trust contribution ratio is determined by the following formula:

[0065]

[0066] Where, is the trust contribution ratio, x is the weather influencing factor, y is the route influencing factor, w1 is the first weight coefficient, w2 is the second weight coefficient, λ is the total length of the route traveled by the current transport vehicle within the preset time period, t1 is the starting time point of the preset time period, and t2 is the ending time point of the preset time period.

[0067] In one embodiment of the present invention, determining the comprehensive temporal characteristics of the current transport vehicle based on the video stream data includes:

[0068] Performing feature extraction on the image data of the current transport vehicle at the multiple different moments to obtain a first feature and a second feature of the current transport vehicle; wherein the first feature is used to characterize a lateral feature of the current transport vehicle, and the second feature is used to characterize a longitudinal feature of the current transport vehicle;

[0069] The first feature and the second feature are superimposed to obtain the comprehensive time series feature.

[0070] In one embodiment of the present invention, after inputting the fusion features into a preset neural network prediction model to obtain the current operating status of the transport vehicle, the following steps are performed:

[0071] Get the running status of the transport vehicle closest to the current transport vehicle;

[0072] Selectively enter a risk control process based on the operating status of the preceding transport vehicle that is closest to the current transport vehicle and the operating status of the current transport vehicle;

[0073] When entering the risk control process, the risk coefficient is determined according to the risk dynamic rating equation group;

[0074] Based on the risk factor, the final operating status of the transport vehicle is determined.

[0075] In one embodiment of the present invention, the operating state includes a normal state and an abnormal state;

[0076] The selectively entering the risk control process based on the operating status of the preceding transport vehicle closest to the current transport vehicle and the operating status of the current transport vehicle includes:

[0077] Determining whether the current operating state of the transport vehicle is an abnormal state;

[0078] If yes, then enter the risk control process;

[0079] If not, when the operating status of the front transport vehicle that is closest to the current transport vehicle is abnormal, the risk control process is entered.

[0080] In one embodiment of the present invention, the risk dynamic rating equation group is constructed by the following formula:

[0081]

[0082]

[0083] Where N is the risk factor, t1 is the starting time point of the preset time period, t2 is the ending time point of the preset time period, Δt is the time interval of data sampling, w(t i ) is the time weight coefficient, s th is the preset safety distance, m g (t i ) is the influence coefficient of the vehicle's own state on distance judgment, s(t i ) at t i The distance between the current transport vehicle and the nearest transport vehicle ahead at the moment, is the correction coefficient under the same road conditions, u(x) is the unit step function, a is the acceleration of the current transport vehicle, and a max is the maximum acceleration of the current transport vehicle, v is the speed of the current transport vehicle, v0 is the expected speed of the current transport vehicle, is the acceleration index, s * (v, Δv) is the desired safe following distance, s0 is the minimum safe distance, T is the driver's preset reaction time, Δv is the speed difference between the vehicle and the preceding vehicle, b is the maximum deceleration, s min Vehicles maintain a minimum safe distance.

[0084] In one embodiment of the present invention, after determining the final operating state of the transport vehicle based on the risk coefficient, the method further includes:

[0085] Determining a combination of risk warning information based on the risk coefficient, and sending the combination of risk warning information to the vehicle end;

[0086] Among them, the risk warning information includes voice prompts, single driver's seat vibration, multiple driver's seat vibrations, and single passenger seat vibration.

[0087] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on a device for monitoring and measuring the condition of a mine transport vehicle. In other embodiments of the present invention, a device for monitoring and measuring the condition of a mine transport vehicle may include more or fewer components than illustrated, or may combine or separate certain components, or may have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0088] The information interaction, execution process, etc. between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For specific contents, please refer to the description in the embodiment of the method of the present invention and will not be repeated here.

[0089] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a method for monitoring the status of a mine transport vehicle in any embodiment of the present invention is implemented.

[0090] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor executes a mine transport vehicle status monitoring method according to any embodiment of the present invention.

[0091] Specifically, a system or device equipped with a storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program codes stored in the storage medium.

[0092] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.

[0093] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0094] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0095] In addition, it can be understood that the program code read from the storage medium is written into the memory provided in the expansion board inserted into the computer or into the memory provided in the expansion module connected to the computer, and then based on the instructions of the program code, the CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above embodiments.

[0096] It should be noted that, in this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0097] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk, etc. Various media that can store program codes.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for monitoring the status of a mine transport vehicle, characterized in that: The method is applied to the cloud, and includes: Obtaining video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different times, and the operating parameter data includes speed, braking distance, brake pedal travel, and tire pressure; Determining the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determining the motion characteristics of the current transport vehicle based on the operating parameter data; Fusing the comprehensive time series features and the motion features according to a trust contribution ratio to obtain a fused feature; wherein the trust contribution ratio is determined based on a weather influencing factor and a route influencing factor; Inputting the fusion features into a preset neural network prediction model to obtain the current operating status of the transport vehicle; The trust contribution ratio is determined by the following formula: Where, is the trust contribution ratio, x is the weather influencing factor, y is the route influencing factor, w1 is the first weight coefficient, w2 is the second weight coefficient, λ is the total length of the route traveled by the current transport vehicle within the preset time period, t1 is the starting time point of the preset time period, and t2 is the ending time point of the preset time period.

2. The method according to claim 1, characterized in that Determine the comprehensive time series characteristics of the current transport vehicle based on the video stream data, including: Performing feature extraction on the image data of the current transport vehicle at the multiple different moments to obtain a first feature and a second feature of the current transport vehicle; wherein the first feature is used to characterize a lateral feature of the current transport vehicle, and the second feature is used to characterize a longitudinal feature of the current transport vehicle; The first feature and the second feature are superimposed to obtain the comprehensive time series feature.

3. The method according to claim 2, characterized in that After inputting the fusion features into a preset neural network prediction model to obtain the current operating status of the transport vehicle, the method includes: Get the running status of the transport vehicle closest to the current transport vehicle; Selectively entering a risk control process based on the operating status of the preceding transport vehicle closest to the current transport vehicle and the operating status of the current transport vehicle; When entering the risk control process, the risk coefficient is determined according to the risk dynamic rating equation group; Based on the risk factor, the final operating status of the transport vehicle is determined.

4. The method according to claim 3, characterized in that The operating state includes a normal state and an abnormal state; The selectively entering the risk control process based on the operating status of the preceding transport vehicle closest to the current transport vehicle and the operating status of the current transport vehicle includes: Determining whether the current operating state of the transport vehicle is an abnormal state; If yes, then enter the risk control process; If not, when the operating status of the front transport vehicle that is closest to the current transport vehicle is abnormal, the risk control process is entered.

5. The method according to claim 4, characterized in that The risk dynamic rating equation group is constructed by the following formula: Where N is the risk factor, t1 is the starting time point of the preset time period, t2 is the ending time point of the preset time period, Δt is the time interval of data sampling, w(t i ) is the time weight coefficient, s th is the preset safety distance, m g (t i ) is the influence coefficient of the vehicle's own state on distance judgment, s(t i ) at t i The distance between the current transport vehicle and the nearest transport vehicle ahead at the moment, is the correction coefficient under the same road conditions, u(x) is the unit step function, a is the acceleration of the current transport vehicle, and a max is the maximum acceleration of the current transport vehicle, v is the speed of the current transport vehicle, v0 is the expected speed of the current transport vehicle, is the acceleration index, s * (v, Δv) is the desired safe following distance, s0 is the minimum safe distance, T is the driver's preset reaction time, Δv is the speed difference between the vehicle and the preceding vehicle, b is the maximum deceleration, s min Vehicles maintain a minimum safe distance.

6. The method according to claim 3, characterized in that After determining the final operating state of the transport vehicle based on the risk factor, the method further includes: Determining a combination of risk warning information based on the risk coefficient, and sending the combination of risk warning information to the vehicle end; Among them, the risk warning information includes voice prompts, single driver's seat vibration, multiple driver's seat vibrations, and single passenger seat vibration.

7. A mine transport vehicle status monitoring device, characterized in that: include: an acquisition module, configured to acquire video stream data and operating parameter data of the current transport vehicle within a preset time period; wherein the video stream data includes image data of the current transport vehicle at multiple different moments, and the operating parameter data includes speed, braking distance, brake pedal travel, and tire pressure; a first data processing module, configured to determine the comprehensive time series characteristics of the current transport vehicle based on the video stream data, and determine the motion characteristics of the current transport vehicle based on the operating parameter data; a second data processing module, configured to perform feature fusion on the comprehensive time series features and the motion features according to a trust contribution ratio to obtain a fused feature; wherein the trust contribution ratio is determined based on a weather influencing factor and a route influencing factor; A third data processing module is used to input the fusion features into a preset neural network prediction model to obtain the current operating status of the transport vehicle; The trust contribution ratio is determined by the following formula: Where, is the trust contribution ratio, x is the weather influencing factor, y is the route influencing factor, w1 is the first weight coefficient, w2 is the second weight coefficient, λ is the total length of the route traveled by the current transport vehicle within the preset time period, t1 is the starting time point of the preset time period, and t2 is the ending time point of the preset time period.

8. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

Images