Internal combustion locomotive data anomaly detection method, device and system, computer equipment, storage medium and computer program product

Abstract

The invention relates to an internal combustion locomotive data anomaly detection method, device and system, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring current video data of a to-be-detected diesel locomotive; respectively extracting a current image feature, a current video feature and a current structural feature of the current video data, and taking the current image feature, the current video feature and the current structural feature as current data features of the current video data; the current structural features are obtained based on element features of a target object contained in the current video data; determining an abnormal feature, a trend feature and an association feature corresponding to the current data feature; and inputting the abnormal features, the trend features and the correlation features into a trained anomaly detection model to obtain a target anomaly detection result of the to-be-detected diesel locomotive. By adopting the method, the detection accuracy of the data exception of the diesel locomotive can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, system, computer equipment, computer-readable storage medium, and computer program product for detecting anomaly in diesel locomotive data. Background Art

[0002] At present, in order to improve the safety of diesel locomotive operation, it is crucial to detect data anomalies on diesel locomotives.

[0003] Traditionally, manual detection has been used to detect data anomalies in diesel locomotives. However, this manual detection method is subjective and prone to errors, resulting in low accuracy in detecting data anomalies in diesel locomotives. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, system, computer equipment, computer-readable storage medium and computer program product for detecting anomalies in diesel locomotive data, which can improve the detection accuracy of anomalies in diesel locomotive data, in order to address the above technical problems.

[0005] In a first aspect, the present application provides a method for detecting anomaly in diesel locomotive data, the method comprising:

[0006] Acquire current video data of the diesel locomotive to be detected; the current video data includes facial video data and driving behavior video data of the driver of the diesel locomotive to be detected, and environmental video data of the diesel locomotive to be detected;

[0007] extracting current image features, current video features, and current structural features of the current video data respectively, and using the current image features, the current video features, and the current structural features as current data features of the current video data; the current structural features are obtained based on element features of the target object contained in the current video data;

[0008] Determining abnormal features, trend features, and correlation features corresponding to the current data features;

[0009] The abnormality feature, the trend feature and the correlation feature are input into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected.

[0010] In one embodiment, respectively extracting the current image features, the current video features, and the current structural features of the current video data includes:

[0011] extracting current image data from the current video data;

[0012] Performing feature extraction processing on the current image data to obtain image visual features of the current image data;

[0013] Performing feature extraction again on the visual features of the image to obtain semantic features of the current image data;

[0014] The image visual features and the semantic features are fused to obtain the current image features.

[0015] In one embodiment, the extracting the current image features, the current video features, and the current structural features of the current video data respectively further includes:

[0016] Performing a time-domain convolution process on the current video data to obtain a time-domain feature of the current video data, and performing a spatial convolution process on the current video data to obtain a spatial feature of the current video data;

[0017] Fusing the temporal features and the spatial features to obtain local temporal and spatial features of the current video data;

[0018] The global spatiotemporal features of the current video data are extracted, and the local spatiotemporal features and the global spatiotemporal features are fused to obtain the current video features.

[0019] In one embodiment, the extracting the current image features, the current video features, and the current structural features of the current video data respectively further includes:

[0020] Performing target detection processing on the current video data to obtain image data of the target object contained in the current video data;

[0021] Performing feature extraction processing on the image data of the target object to obtain element features of the target object;

[0022] The element features of each target object are fused to obtain the current structured features.

[0023] In one embodiment, the inputting of the abnormality feature, the trend feature, and the correlation feature into a trained abnormality detection model to obtain a target abnormality detection result of the diesel locomotive to be detected includes:

[0024] Inputting the abnormality feature, the trend feature, and the correlation feature into the trained abnormality detection model to obtain a target prediction probability of the diesel locomotive to be detected under each preset abnormality detection result;

[0025] From the preset abnormality detection results, the preset abnormality detection results whose target prediction probability is greater than the preset prediction probability are screened out as the target abnormality detection results of the diesel locomotive to be detected.

[0026] In one embodiment, the inputting of the abnormality feature, the trend feature, and the correlation feature into the trained abnormality detection model to obtain the target prediction probability of the diesel locomotive to be detected under each preset abnormality detection result includes:

[0027] Inputting the abnormality feature into the trained abnormality detection model to obtain a first predicted probability of the diesel locomotive to be detected under each preset abnormality detection result;

[0028] Inputting the trend feature into the trained anomaly detection model to obtain a second predicted probability of the diesel locomotive to be detected under each preset anomaly detection result;

[0029] Inputting the associated features into the trained anomaly detection model to obtain a third predicted probability of the diesel locomotive to be detected under each preset anomaly detection result;

[0030] The first prediction probability, the second prediction probability and the third prediction probability of the diesel locomotive to be detected under each preset abnormal detection result are summed respectively to obtain the target prediction probability of the diesel locomotive to be detected under each preset abnormal detection result.

[0031] In a second aspect, the present application further provides a device for detecting anomaly in data of a diesel locomotive, the device comprising:

[0032] a data acquisition module for acquiring current video data of the diesel locomotive to be detected; the current video data includes facial video data and driving behavior video data of the driver of the diesel locomotive to be detected, and environmental video data of the diesel locomotive to be detected;

[0033] a feature extraction module, configured to respectively extract current image features, current video features, and current structural features of the current video data, and use the current image features, the current video features, and the current structural features as current data features of the current video data; the current structural features are obtained based on element features of a target object contained in the current video data;

[0034] A feature determination module, configured to determine abnormal features, trend features, and correlation features corresponding to the current data features;

[0035] The model detection module is used to input the abnormality feature, the trend feature and the correlation feature into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected.

[0036] In a third aspect, the present application further provides a diesel locomotive data anomaly detection system, the system comprising: a data acquisition module, a data preprocessing module, a data exploration module, and a multidimensional data processing module; the data preprocessing module is communicatively connected to the data acquisition module and the data exploration module, respectively, and the data exploration module is communicatively connected to the multidimensional data processing module;

[0037] The data acquisition module is used to obtain current video data of the diesel locomotive to be detected and send the current video data to the data preprocessing module; the current video data includes facial video data and driving behavior video data of the driver of the diesel locomotive to be detected, and environmental video data of the diesel locomotive to be detected;

[0038] The data preprocessing module is used to extract current image features, current video features, and current structural features of the current video data, respectively, and use the current image features, the current video features, and the current structural features as current data features of the current video data, and send the current data features to the data exploration module; the current structural features are obtained based on the element features of the target object contained in the current video data;

[0039] The data exploration module is configured to determine abnormal features, trend features, and correlation features corresponding to the current data features, and send the abnormal features, trend features, and correlation features to the multidimensional data processing module;

[0040] The multi-dimensional data processing module is used to input the abnormality feature, the trend feature and the correlation feature into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected.

[0041] In a fourth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0042] Acquire current video data of the diesel locomotive to be detected; the current video data includes facial video data and driving behavior video data of the driver of the diesel locomotive to be detected, and environmental video data of the diesel locomotive to be detected;

[0043] extracting current image features, current video features, and current structural features of the current video data respectively, and using the current image features, the current video features, and the current structural features as current data features of the current video data; the current structural features are obtained based on element features of the target object contained in the current video data;

[0044] Determining abnormal features, trend features, and correlation features corresponding to the current data features;

[0045] The abnormality feature, the trend feature and the correlation feature are input into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected.

[0046] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0047] Acquire current video data of the diesel locomotive to be detected; the current video data includes facial video data and driving behavior video data of the driver of the diesel locomotive to be detected, and environmental video data of the diesel locomotive to be detected;

[0048] extracting current image features, current video features, and current structural features of the current video data respectively, and using the current image features, the current video features, and the current structural features as current data features of the current video data; the current structural features are obtained based on element features of the target object contained in the current video data;

[0049] Determining abnormal features, trend features, and correlation features corresponding to the current data features;

[0050] The abnormality feature, the trend feature and the correlation feature are input into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected.

[0051] In a sixth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0052] Acquire current video data of the diesel locomotive to be detected; the current video data includes facial video data and driving behavior video data of the driver of the diesel locomotive to be detected, and environmental video data of the diesel locomotive to be detected;

[0053] extracting current image features, current video features, and current structural features of the current video data respectively, and using the current image features, the current video features, and the current structural features as current data features of the current video data; the current structural features are obtained based on element features of the target object contained in the current video data;

[0054] Determining abnormal features, trend features, and correlation features corresponding to the current data features;

[0055] The abnormality feature, the trend feature and the correlation feature are input into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected.

[0056] The above-mentioned diesel locomotive data anomaly detection method, apparatus, system, computer equipment, storage medium and computer program product first obtain the facial video data and driving behavior video data of the driver of the diesel locomotive to be detected, as well as the environmental video data of the diesel locomotive to be detected, all of which are used as the current video data of the diesel locomotive to be detected, and then respectively extract the current image features, current video features and current structured features of the current video data, and use the current image features, current video features and current structured features as the current data features of the current video data. Then, the abnormal features, trend features and correlation features corresponding to the current data features are determined. Finally, the abnormal features, trend features and correlation features are input into the trained anomaly detection model to obtain the target anomaly detection results of the diesel locomotive to be detected. In this way, in the process of data anomaly detection for diesel locomotives, by obtaining multiple aspects of current video data of the diesel locomotive to be detected and performing multi-dimensional feature extraction to obtain the current data features of the current video data, and then through in-depth feature analysis, further determine the abnormal features, trend features and correlation features corresponding to the current data features, and combine with the trained anomaly detection model, the target anomaly detection results of the diesel locomotive to be detected can be obtained more accurately, thereby improving the accuracy of determining the target anomaly detection results of the diesel locomotive to be detected, which is conducive to improving the detection accuracy of diesel locomotive data anomalies; moreover, the entire process does not require human intervention, avoiding the subjective factors in the manual detection method, which is prone to errors and leads to the defect of low detection accuracy of diesel locomotive data anomalies, thereby improving the detection accuracy of diesel locomotive data anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0058] Figure 1 1 is a flow chart of a method for detecting abnormality in diesel locomotive data according to one embodiment;

[0059] Figure 2 A schematic flow chart of a method for detecting abnormality in diesel locomotive data in another embodiment;

[0060] Figure 3 Schematic diagram of the structure of a diesel locomotive data anomaly detection system in one embodiment;

[0061] Figure 4 Schematic diagram of the structure of a data preprocessing module in one embodiment;

[0062] Figure 5 is a structural block diagram of a device for detecting abnormal data of a diesel locomotive in one embodiment;

[0063] Figure 6 A schematic structural diagram of a diesel locomotive data anomaly detection system in another embodiment;

[0064] Figure 7 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0066] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0067] In an exemplary embodiment, Figure 1As shown, a method for detecting anomalies in diesel locomotive data is provided. This embodiment uses the method applied to a server as an example for illustration; it is understood that the method can also be applied to a terminal, or to a system including a terminal and a server, and implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, and tablet computers; the server can be implemented as an independent server or a server cluster consisting of multiple servers. In this embodiment, the method includes the following steps:

[0068] Step S101, obtaining current video data of the diesel locomotive to be detected; the current video data includes facial video data and driving behavior video data of the driver in the diesel locomotive to be detected, and environmental video data of the diesel locomotive to be detected.

[0069] The diesel locomotive to be inspected refers to a diesel locomotive that needs to be inspected for abnormalities.

[0070] The current video data includes facial video data and driving behavior video data of the driver of the diesel locomotive to be detected, as well as environmental video data of the diesel locomotive to be detected.

[0071] Among them, facial video data refers to video data obtained by shooting the driver's face, including video data such as the driver's facial expression, head posture, eye gaze direction, blinking frequency, etc.

[0072] Driving behavior video data refers to video data obtained by recording the various operating behaviors of the driver while driving a diesel locomotive, including the driver's operating actions on locomotive control handles, buttons, switches and other equipment, as well as other related body movements.

[0073] The environmental video data refers to the video data obtained by shooting the external and internal driving environment of the locomotive.

[0074] For example, the server determines the current running time of the candidate diesel locomotives, and selects the candidate diesel locomotives with a current running time greater than the preset running time from each candidate diesel locomotive as the diesel locomotive to be detected; then, the server obtains the anti-drowsiness data of the diesel locomotive to be detected (including the driver's facial video data, driving behavior video data, physiological state data, and the diesel locomotive's motion data, direction data, acceleration data, position data, lane change behavior data), running gear detection data (including axle temperature rise alarm data, axle temperature over-temperature alarm data, running gear impact alarm data, running gear equipment failure data), smart bed blanket data (including check-in time, check-out time, sleep quality, call time, bed time, sleep time), etc. The server uses the following data as the current locomotive data of the diesel locomotive: the data of the driver's facial video data and driving behavior video data of the diesel locomotive to be tested, as well as the environmental video data of the diesel locomotive to be tested, based on the data type of the current locomotive data:

[0075] Step S102, respectively extract the current image features, current video features and current structural features of the current video data, and use the current image features, current video features and current structural features as the current data features of the current video data; the current structural features are obtained based on the element features of the target object contained in the current video data.

[0076] The current image feature refers to a feature vector corresponding to each frame of the image in the current video data.

[0077] The current video feature refers to the feature vector corresponding to the current video data.

[0078] The current structured feature refers to a feature obtained after a structured analysis of the current video data, and is obtained based on the element features of the target object contained in the current video data.

[0079] The current data features include current image features, current video features, and current structural features of the current video data.

[0080] The target object refers to various target objects in the current video data, such as people, moving objects, and static objects.

[0081] Among them, the element features include feature vectors corresponding to the attributes, actions, quantity, color, posture and other data of the target object.

[0082] Exemplarily, the server performs feature extraction processing on the current video data to obtain the current image features, current video features, and element features of the target object contained in the current video data; then, the server obtains the current structured features of the current video data based on the element features of the target object contained in the current video data; then, the server uses the current image features, current video features, and current structured features of the current video data as the current data features of the current video data.

[0083] Step S103: determining abnormal features, trend features, and correlation features corresponding to the current data features.

[0084] Among them, abnormal features refer to sub-data features in current data features that deviate significantly from historical normal data features.

[0085] Among them, the trend feature is used to indicate the degree of change corresponding to the current data feature.

[0086] Among them, the association feature is used to represent the intrinsic connection between each sub-data feature in the current data feature, such as the intrinsic connection between the current image feature and the current video feature, the intrinsic connection between the current image feature and the current structured feature, and the intrinsic connection between the current video feature and the current structured feature.

[0087] Exemplarily, the server obtains multiple historical normal data features corresponding to the diesel locomotive to be detected, and averages the multiple historical normal data features to obtain the average normal data feature corresponding to the diesel locomotive to be detected; then, the server determines the Euclidean distance between each sub-data feature in the current data feature and the average normal data feature as the current Euclidean distance corresponding to each sub-data feature, and from each sub-data feature, screens out the sub-data feature whose corresponding current Euclidean distance is greater than the distance threshold as the abnormal feature corresponding to the current data feature; then, the server determines the historical data feature corresponding to the current data feature, and determines the difference between the current data feature and the historical data feature as the trend feature corresponding to the current data feature; then, the server determines the correlation coefficient between each sub-data feature in the current data feature as the association feature corresponding to the current data feature.

[0088] In step S104, the abnormal features, trend features and correlation features are input into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected.

[0089] The trained anomaly detection model refers to a network model that can obtain target anomaly detection results corresponding to the diesel locomotive to be detected, such as a convolutional neural network model.

[0090] Among them, the target anomaly detection result is used to indicate the anomaly type corresponding to the diesel locomotive to be detected, including locomotive component failure, abnormal operating parameters, improper driver operation, improper crew operation, no abnormal situation, etc.

[0091] Exemplarily, the server preprocesses the abnormal features, trend features, and correlation features respectively to obtain preprocessed abnormal features, preprocessed trend features, and preprocessed correlation features; then, the server inputs the preprocessed abnormal features, preprocessed trend features, and preprocessed correlation features into the trained abnormality detection model, and determines the target abnormality detection results of the diesel locomotive to be detected from each preset abnormality detection result through the trained abnormality detection model.

[0092] In the above-mentioned diesel locomotive data anomaly detection method, facial video data and driving behavior video data of the driver of the diesel locomotive to be detected, as well as environmental video data of the diesel locomotive to be detected, are first obtained, all of which are used as current video data of the diesel locomotive to be detected. Then, the current image features, current video features and current structural features of the current video data are extracted respectively, and the current image features, current video features and current structural features are all used as current data features of the current video data. Then, the abnormal features, trend features and correlation features corresponding to the current data features are determined. Finally, the abnormal features, trend features and correlation features are input into the trained anomaly detection model to obtain the target anomaly detection results of the diesel locomotive to be detected. In this way, in the process of data anomaly detection for diesel locomotives, by obtaining multiple aspects of current video data of the diesel locomotive to be detected and performing multi-dimensional feature extraction to obtain the current data features of the current video data, and then through in-depth feature analysis, further determine the abnormal features, trend features and correlation features corresponding to the current data features, and combine with the trained anomaly detection model, the target anomaly detection results of the diesel locomotive to be detected can be obtained more accurately, thereby improving the accuracy of determining the target anomaly detection results of the diesel locomotive to be detected, which is conducive to improving the detection accuracy of diesel locomotive data anomalies; moreover, the entire process does not require human intervention, avoiding the subjective factors in the manual detection method, which is prone to errors and leads to the defect of low detection accuracy of diesel locomotive data anomalies, thereby improving the detection accuracy of diesel locomotive data anomalies.

[0093] In an exemplary embodiment, the above-mentioned step S102 extracts the current image features, current video features and current structural features of the current video data respectively, and specifically includes the following contents: extracting the current image data in the current video data; performing feature extraction processing on the current image data to obtain the image visual features of the current image data; performing feature extraction processing on the image visual features again to obtain the semantic features of the current image data; and performing fusion processing on the image visual features and the semantic features to obtain the current image features.

[0094] The current image data refers to image data of a certain frame or a series of frames extracted from the current video data.

[0095] The image visual features are used to represent a set of feature vectors that can describe the visual content of the image after feature extraction of the current image data.

[0096] The semantic feature is used to represent a feature vector that can reflect the semantic information in the current image data.

[0097] Exemplarily, the server extracts the current image data from the current video data at a preset frame rate (for example, 10 frames of image are extracted per second); then, the server performs color feature extraction processing, texture feature extraction processing, and shape feature extraction processing on the current image data to obtain the color features, texture features, and shape features of the current image data, and combines the color features, texture features, and shape features of the current image data in a preset combination manner to obtain the image visual features of the current image data; then, the server inputs the image visual features of the current image data into the trained semantic feature extraction model, and performs feature extraction processing on the image visual features again through the trained semantic feature extraction model to obtain the semantic features of the current image data; then, the server determines a first weight corresponding to the image visual feature and a second weight corresponding to the semantic feature, and sums the image visual features and the semantic features according to the first weight and the second weight to obtain the current image features.

[0098] In this embodiment, by performing multiple feature extraction processes on the current image data, the image visual features and semantic features of the current image data are obtained, so that the features corresponding to the current image data can be described from different angles; moreover, the image visual features and semantic features are integrated so that the current image features can simultaneously include the detailed information and content understanding of the image, thereby being able to describe the current image data more comprehensively and accurately.

[0099] In an exemplary embodiment, the above-mentioned step S102 extracts the current image features, current video features and current structural features of the current video data respectively, which specifically includes the following contents: performing time domain convolution processing on the current video data to obtain the time domain features of the current video data, and performing spatial convolution processing on the current video data to obtain the spatial features of the current video data; fusing the time domain features and the spatial features to obtain the local spatiotemporal features of the current video data; extracting the global spatiotemporal features of the current video data, and fusing the local spatiotemporal features and the global spatiotemporal features to obtain the current video features.

[0100] Among them, the time domain feature refers to the feature of the local area in the current video data in the time dimension, such as the movement speed of the object in the current video data.

[0101] The spatial feature refers to the feature of a local area in the current video data in the spatial dimension, such as the spatial relationship between objects in the current video data.

[0102] Among them, the local spatiotemporal features refer to the features of the local area in the current video data in the spatiotemporal dimension.

[0103] Among them, the global spatiotemporal features refer to the features of all frames in the current video data in the spatiotemporal dimension.

[0104] Exemplarily, the server extracts key frames from the current video data; then, the server inputs the key frames into a time domain feature extraction model, performs time domain convolution processing on the key frames through the time domain feature extraction model, and obtains the time domain features of the key frames as the time domain features of the current video data; then, the server inputs the key frames into a spatial feature extraction model, performs spatial convolution processing on the current video data through the spatial feature extraction model, and obtains the spatial features of the key frames as the spatial features of the current video data; then, the server splices the time domain features and spatial features according to a preset splicing order to obtain local spatiotemporal features of the current video data; then, the server extracts all frames from the current video data; then, the server extracts the spatiotemporal features of all frames in the current video data, and averages the spatiotemporal features of all frames in the current video data to obtain global spatiotemporal features of the current video data; then, the server determines a first attention weight corresponding to the local spatiotemporal features, and a second attention weight corresponding to the global spatiotemporal features, and fuses the local spatiotemporal features and the global spatiotemporal features according to the first attention weight and the second attention weight to obtain the current video features.

[0105] In this embodiment, by performing time domain convolution processing and spatial convolution processing on the current video data, the local spatiotemporal features of the current video data are obtained, and combined with the global spatiotemporal features of the current video data, it is possible to consider the overall information of the video while taking into account local details, thereby being able to grasp the content of the video more comprehensively, which is conducive to improving the accuracy of determining the current video features.

[0106] In an exemplary embodiment, the above step S102 extracts the current image features, current video features and current structural features of the current video data respectively, which specifically includes the following contents: performing target detection processing on the current video data to obtain the image data of the target object contained in the current video data; performing feature extraction processing on the image data of the target object to obtain the element features of the target object; performing fusion processing on the element features of each target object to obtain the current structural features.

[0107] Exemplarily, the server performs denoising on the current video data to obtain denoised video data; then, the server determines the current application scenario of the denoised video data, and queries the correspondence between the application scenario and the target detection model to obtain the target detection model corresponding to the denoised video data; then, the server inputs the denoised video data into the target detection model corresponding to the denoised video data, performs target detection on the current video data through the target detection model, and obtains the image data of the target object contained in the current video data; then, the server inputs the image data of the target object into the feature extraction model, performs feature extraction on the image data of the target object through the feature extraction model, and obtains the element features of the target object; then, the server performs feature alignment on the element features of each target object to obtain the processed element features of each target object, and performs weighted summation on the processed element features of each target object to obtain the current structured features.

[0108] In this embodiment, by performing target detection processing, feature extraction processing and fusion processing on the current video data, a more comprehensive and accurate current structured feature can be obtained for the current video data, which is conducive to improving the accuracy of determining the current structured feature, and thus improving the accuracy and reliability of subsequent analysis tasks.

[0109] In an exemplary embodiment, the above-mentioned step S104, in which the abnormal features, trend features, and correlation features are input into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected, specifically includes the following contents: the abnormal features, trend features, and correlation features are input into the trained abnormality detection model to obtain the target prediction probability of the diesel locomotive to be detected under each preset abnormality detection result; from each preset abnormality detection result, the preset abnormality detection result having a target prediction probability greater than the preset prediction probability is screened out as the target abnormality detection result of the diesel locomotive to be detected.

[0110] Among them, the preset abnormal detection results refer to the pre-set abnormal detection results, including locomotive component failure, abnormal operating parameters, improper driver operation, improper crew operation, no abnormal situation, etc.

[0111] Among them, the target prediction probability refers to the comprehensive prediction probability corresponding to the preset anomaly detection result determined based on anomaly characteristics, trend characteristics and correlation characteristics.

[0112] The preset prediction probability refers to a pre-set prediction probability, such as 80%. It should be noted that the preset prediction probability depends on the circumstances.

[0113] Exemplarily, the server preprocesses the abnormal features, trend features and correlation features respectively to obtain preprocessed abnormal features, preprocessed trend features and preprocessed correlation features; then, the server inputs the preprocessed abnormal features, preprocessed trend features and preprocessed correlation features into the trained abnormality detection model, and obtains the target prediction probability of the diesel locomotive to be detected under each preset abnormality detection result through the trained abnormality detection model; then, the server filters out the preset abnormality detection results whose target prediction probability is greater than the preset prediction probability from each preset abnormality detection result, and uses these preset abnormality detection results as the target abnormality detection results of the diesel locomotive to be detected.

[0114] In this embodiment, by inputting abnormal features, trend features, and correlation features into the anomaly detection model, the status of the diesel locomotive can be evaluated from multiple angles. Combining these three types of features can more comprehensively and accurately capture potential abnormal conditions of the diesel locomotive, avoid missed detections or false detections that may result from single feature analysis, and facilitate accurate judgment of whether there is an abnormality in the locomotive.

[0115] In an exemplary embodiment, abnormal features, trend features and correlation features are input into a trained abnormality detection model to obtain a target prediction probability of the diesel locomotive to be detected under each preset abnormality detection result, specifically including the following contents: abnormal features are input into the trained abnormality detection model to obtain a first prediction probability of the diesel locomotive to be detected under each preset abnormality detection result; trend features are input into the trained abnormality detection model to obtain a second prediction probability of the diesel locomotive to be detected under each preset abnormality detection result; correlation features are input into the trained abnormality detection model to obtain a third prediction probability of the diesel locomotive to be detected under each preset abnormality detection result; the first prediction probability, second prediction probability and third prediction probability of the diesel locomotive to be detected under each preset abnormality detection result are summed up respectively to obtain the target prediction probability of the diesel locomotive to be detected under each preset abnormality detection result.

[0116] The first predicted probability refers to the predicted probability corresponding to the preset anomaly detection result determined based on the abnormal feature.

[0117] The second predicted probability refers to the predicted probability corresponding to the preset anomaly detection result determined based on the trend feature.

[0118] The third predicted probability refers to the predicted probability corresponding to the preset anomaly detection result determined based on the associated features.

[0119] Exemplarily, the server preprocesses the abnormality feature, trend feature and correlation feature respectively to obtain preprocessed abnormality feature, preprocessed trend feature and preprocessed correlation feature; then, the server inputs the preprocessed abnormality feature into the trained abnormality detection model to obtain a first prediction probability of the diesel locomotive to be detected under each preset abnormality detection result; then, the server inputs the preprocessed trend feature into the trained abnormality detection model to obtain a second prediction probability of the diesel locomotive to be detected under each preset abnormality detection result; then, the server inputs the preprocessed correlation feature into the trained abnormality detection model to obtain a third prediction probability of the diesel locomotive to be detected under each preset abnormality detection result; then, the server sums the first prediction probability, second prediction probability and third prediction probability of the diesel locomotive to be detected under each preset abnormality detection result according to the first preset weight corresponding to the preprocessed abnormality feature, the second preset weight corresponding to the preprocessed trend feature and the third preset weight corresponding to the preprocessed correlation feature to obtain a target prediction probability of the diesel locomotive to be detected under each preset abnormality detection result.

[0120] In this embodiment, by inputting abnormal features, trend features and correlation features into the model for analysis respectively, various potential abnormal situations in the operation of the diesel locomotive can be comprehensively and deeply explored, avoiding the possibility that a single feature may not be able to fully present the complex operating status of the diesel locomotive, which may easily cause the omission of abnormal information, and is conducive to improving the comprehensiveness of abnormality detection.

[0121] In an exemplary embodiment, Figure 2 As shown, another method for detecting anomalies in diesel locomotive data is provided. This method is described by taking its application to a server as an example. Specifically, the method includes the following steps:

[0122] Step S201, obtaining current video data of the diesel locomotive to be detected; the current video data includes facial video data and driving behavior video data of the driver in the diesel locomotive to be detected, and environmental video data of the diesel locomotive to be detected.

[0123] Step S202, extract the current image data from the current video data; perform feature extraction processing on the current image data to obtain the image visual features of the current image data; perform feature extraction processing on the image visual features again to obtain the semantic features of the current image data; perform fusion processing on the image visual features and the semantic features to obtain the current image features.

[0124] Step S203: perform time domain convolution processing on the current video data to obtain the time domain features of the current video data, and perform spatial convolution processing on the current video data to obtain the spatial features of the current video data; fuse the time domain features and the spatial features to obtain the local spatiotemporal features of the current video data; extract the global spatiotemporal features of the current video data, and fuse the local spatiotemporal features with the global spatiotemporal features to obtain the current video features.

[0125] Step S204: perform target detection processing on the current video data to obtain image data of the target object contained in the current video data; perform feature extraction processing on the image data of the target object to obtain element features of the target object; perform fusion processing on the element features of each target object to obtain the current structured features.

[0126] In step S205 , the current image feature, the current video feature, and the current structural feature are all used as the current data feature of the current video data.

[0127] Step S206: Determine the abnormal features, trend features, and correlation features corresponding to the current data features.

[0128] Step S207 : Input the abnormal features into the trained abnormality detection model to obtain a first predicted probability of the diesel locomotive to be detected under each preset abnormality detection result.

[0129] Step S208 : inputting the trend feature into the trained anomaly detection model to obtain a second predicted probability of the diesel locomotive to be detected under each preset anomaly detection result.

[0130] Step S209 : inputting the associated features into the trained anomaly detection model to obtain a third predicted probability of the diesel locomotive to be detected under each preset anomaly detection result.

[0131] Step S210 , summing up the first prediction probability, the second prediction probability and the third prediction probability of the diesel locomotive to be detected under each preset abnormality detection result, to obtain a target prediction probability of the diesel locomotive to be detected under each preset abnormality detection result.

[0132] Step S211 , from each preset abnormality detection result, a preset abnormality detection result having a target prediction probability greater than a preset prediction probability is selected as a target abnormality detection result of the diesel locomotive to be detected.

[0133] In the above-mentioned method for detecting anomalies in diesel locomotive data, during the process of performing data anomaly detection on a diesel locomotive, the current data features of the current video data are obtained by obtaining various aspects of current video data of the diesel locomotive to be detected and performing multi-dimensional feature extraction. Then, through in-depth feature analysis, the abnormal features, trend features and correlation features corresponding to the current data features are further determined. Combined with the trained anomaly detection model, the target anomaly detection results of the diesel locomotive to be detected can be obtained more accurately, thereby improving the accuracy of determining the target anomaly detection results of the diesel locomotive to be detected, which is conducive to improving the detection accuracy of diesel locomotive data anomalies. Moreover, the entire process does not require manual intervention, avoiding the subjective factors and prone to errors in manual detection, which leads to low detection accuracy of diesel locomotive data anomalies, thereby improving the detection accuracy of diesel locomotive data anomalies.

[0134] In an exemplary embodiment, in order to more clearly illustrate the diesel locomotive data anomaly detection method provided by the embodiment of the present application, the diesel locomotive data anomaly detection method is specifically described below with a specific embodiment. In one embodiment, the present application also provides a multi-dimensional data analysis system for diesel locomotive operation safety. In the process of data anomaly detection of diesel locomotives, it includes a data acquisition module, a data exploration module, a multi-dimensional data processing module, a data preprocessing module, a warning module, a wireless communication module, a data storage module and a display module. The data acquisition module is communicated with the data preprocessing module, and uses the existing camera to realize intelligent analysis of the internal and external environment inspection of the locomotive through the algorithm. After the system analysis is completed, an analysis report can be generated; based on the one-time service operation standard of the diesel locomotive crew, according to the system analysis and data processing, some behaviors of the crew are guided and reminded; through the data interface and artificial intelligence algorithm, the crew's work standardization is checked, and anomalies are warned, and video slice files corresponding to the abnormal items are generated to facilitate tracing and abnormal review. Specifically including the following contents:

[0135] like Figure 3 and Figure 4 As shown, the multidimensional data analysis system for diesel locomotive operation safety includes a data acquisition module, a data exploration module, a multidimensional data processing module, a data preprocessing module, a warning module, a wireless communication module, a data storage module and a display module. The data acquisition module is communicatively connected to the data preprocessing module, the data preprocessing module is communicatively connected to the data exploration module, the data exploration module is communicatively connected to the multidimensional data processing module, the multidimensional data processing module is communicatively connected to the multidimensional data processing module, the multidimensional data processing module is communicatively connected to the warning module, the display module and the wireless communication module are both communicatively connected to the multidimensional data processing module, and the data preprocessing module includes an image feature learning unit, a video content understanding unit and a high-throughput video computing unit.

[0136] The data acquisition module is used to collect all the data of the driver from the process of attendance, on-duty and off-duty; the data exploration module is used to explore the data with descriptive statistics and visualization tools to find outliers, trends or associations in the data; the data preprocessing module is used to preprocess, understand and perform preliminary calculations on the data; the image feature learning unit is used to extract the high-level semantic attributes contained in the image and integrate them into the image visual features, further detect the key objects in the image, and explore the relationship between objects as a finer-grained semantic feature to enhance the image feature learning, and is used to construct an image semantic tree structure from the image to the key object, and then to the finer-grained object segmentation area; the video content understanding unit is to effectively learn the features of the three-dimensional video data by decomposing the time domain convolution and the space domain convolution, and further integrate the global spatiotemporal feature points into the pseudo three-dimensional network structure, so as to The high-throughput video computing unit identifies people, moving objects, static objects, scenes, and behaviors, and performs comprehensive analysis of their attributes, movements, quantity, color, posture, and characteristic identification elements. It outputs structured analysis results and triggers corresponding prompts, warnings, decisions, and processing actions according to pre-agreed analysis and judgment rules. The multi-dimensional data processing module analyzes and calculates collected data, integrates data from existing and under-construction systems, and implements comprehensive diesel locomotive safety protection through data correlation analysis. The warning module issues alarms for violations. The wireless communication module transmits data wirelessly. The data storage module stores analyzed and collected data. The display module intelligently analyzes the collected data and displays conditions that affect driving safety. The data acquisition module collects data including anti-drowsiness data, running gear detection data, smart bed blanket data, transportation safety data, and crew operation data.

[0137] It should be noted that in this embodiment, the data preprocessing module can be an onboard multidimensional data processing device equipped with a Linux operating system, which offers enhanced system stability in industrial environments. The intelligent analysis module supports analysis of four or more video channels, with image resolutions exceeding 1280*720 and analysis latency of no more than one second. It also supports Ethernet, USB (Universal Serial Bus), and RS485 (Recommended Standard 485) interfaces, full 4G (Fourth Generation) mobile communication technology, and Ethernet communication with wireless transmission devices. The wireless communication module is an onboard wireless transmission device, which establishes a train-to-ground communication link, with which the onboard multidimensional data processing device communicates via Ethernet. The onboard wireless transmission device broadcasts real-time train operation data from the train operation monitoring and recording device (LKJ) to the onboard multidimensional data processing device, which serves as the basis for analysis and processing. It also provides a secure data channel for the onboard multidimensional data processing device, enabling data transmission from train to ground.

[0138] Anti-drowsiness data is obtained through one or more of the following: cameras, in-vehicle sensors, physiological parameter monitoring, and driving behavior analysis.

[0139] It should be noted that the anti-drowsiness data acquisition method in this embodiment is as follows: camera: uses facial recognition technology to monitor the driver's eyes and facial expressions to detect signs of fatigue and drowsiness; in-vehicle sensors: detect the vehicle's movement, direction and acceleration to assess whether the driver is in control of the vehicle; physiological parameter monitoring: monitors the driver's physiological state through physiological sensors to detect signs of fatigue in advance; driving behavior analysis: analyzes driving behavior including the vehicle's position on the road, driving speed, and lane change behavior to detect abnormal driving patterns.

[0140] Smart bed blanket data include check-in time, check-out time, sleep quality, call-up time, time in bed, sleep duration, deep sleep duration, number of turns over, and respiratory rate; running gear detection data include axle temperature rise alarm, axle temperature over-temperature alarm, running gear impact alarm, and running gear equipment failure; transportation safety data include crew attendance and departure, disclosure, question bank, overwork, crew plan, personnel database, locomotive garage, registered communication, alcohol testing, personnel status, rest time statistics and other data.

[0141] It should be noted that in this embodiment, all of the above is collected through the data acquisition module, with transportation and safety data acquired through the video surveillance device. The video surveillance device and the data processing module communicate via Ethernet, using the UDP (User Datagram Protocol) protocol. The purpose is to obtain video data from all cameras in the video surveillance device, as well as smoke and temperature sensor data. After establishing a DP connection, the video data is streamed using the RTSP (Real Time Streaming Protocol). The specific process is as follows: query the server for available methods; obtain media description information; establish an RTSP session; request to start data transmission; close the session and exit.

[0142] The specific process of the diesel locomotive data anomaly detection method is as follows:

[0143] S1. The data acquisition module obtains abnormal mental state data and action behavior information of crew members and drivers;

[0144] S2, the data acquisition module analyzes the collected information in real time according to the needs and generates feasibility results, and performs intelligent analysis on video AI (Artificial Intelligence);

[0145] S3. Use descriptive statistics and visualization tools to find outliers, trends, or associations in the data and send the data to the multidimensional data processing module;

[0146] S4. The multi-dimensional data processing module classifies the analysis results and directly reminds the crew members of the data that affects their safe driving in voice and text through the wireless communication module, and at the same time transmits the analysis data to the data storage module for storage.

[0147] In the above-mentioned embodiment, during the process of data anomaly detection on a diesel locomotive, current video data of various aspects of the diesel locomotive to be detected is obtained, and multi-dimensional feature extraction is performed to obtain current data features of the current video data. Then, through in-depth feature analysis, abnormal features, trend features, and correlation features corresponding to the current data features are further determined. Combined with the trained anomaly detection model, a target anomaly detection result of the diesel locomotive to be detected can be obtained more accurately, thereby improving the accuracy of determining the target anomaly detection result of the diesel locomotive to be detected, which is conducive to improving the detection accuracy of diesel locomotive data anomalies. Moreover, the entire process does not require human intervention, avoiding the subjective factors and proneness to errors in manual detection, which leads to low detection accuracy of diesel locomotive data anomalies, thereby improving the detection accuracy of diesel locomotive data anomalies.

[0148] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0149] Based on the same inventive concept, embodiments of the present application also provide a diesel locomotive data anomaly detection device for implementing the aforementioned diesel locomotive data anomaly detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more of the following embodiments of the diesel locomotive data anomaly detection device can be found in the aforementioned limitations of the diesel locomotive data anomaly detection method and will not be further elaborated here.

[0150] In an exemplary embodiment, Figure 5 As shown, a device for detecting anomaly in data of a diesel locomotive is provided, comprising: a data acquisition module 501, a feature extraction module 502, a feature determination module 503 and a model detection module 504, wherein:

[0151] The data acquisition module 501 is used to acquire the current video data of the diesel locomotive to be detected; the current video data includes the facial video data and driving behavior video data of the driver in the diesel locomotive to be detected, and the environmental video data of the diesel locomotive to be detected.

[0152] The feature extraction module 502 is used to extract the current image features, current video features and current structural features of the current video data respectively, and use the current image features, current video features and current structural features as the current data features of the current video data; the current structural features are obtained based on the element features of the target object contained in the current video data.

[0153] The feature determination module 503 is used to determine the abnormal features, trend features and correlation features corresponding to the current data features.

[0154] The model detection module 504 is used to input the abnormality features, trend features and correlation features into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected.

[0155] In an exemplary embodiment, the feature extraction module 502 is also used to extract the current image data from the current video data; perform feature extraction processing on the current image data to obtain the image visual features of the current image data; perform feature extraction processing on the image visual features again to obtain the semantic features of the current image data; and perform fusion processing on the image visual features and the semantic features to obtain the current image features.

[0156] In an exemplary embodiment, the feature extraction module 502 is also used to perform time domain convolution processing on the current video data to obtain the time domain features of the current video data, and to perform space convolution processing on the current video data to obtain the space features of the current video data; to fuse the time domain features and the space features to obtain the local space-time features of the current video data; to extract the global space-time features of the current video data, and to fuse the local space-time features and the global space-time features to obtain the current video features.

[0157] In an exemplary embodiment, the feature extraction module 502 is also used to perform target detection processing on the current video data to obtain image data of the target object contained in the current video data; perform feature extraction processing on the image data of the target object to obtain element features of the target object; and perform fusion processing on the element features of each target object to obtain the current structured features.

[0158] In an exemplary embodiment, the model detection module 504 is further used to input abnormal features, trend features, and correlation features into the trained abnormality detection model to obtain the target prediction probability of the diesel locomotive to be detected under each preset abnormality detection result; from each preset abnormality detection result, the preset abnormality detection result with a target prediction probability greater than the preset prediction probability is screened out as the target abnormality detection result of the diesel locomotive to be detected.

[0159] In an exemplary embodiment, the model detection module 504 is also used to input the abnormality feature into the trained abnormality detection model to obtain the first prediction probability of the diesel locomotive to be detected under each preset abnormality detection result; input the trend feature into the trained abnormality detection model to obtain the second prediction probability of the diesel locomotive to be detected under each preset abnormality detection result; input the associated feature into the trained abnormality detection model to obtain the third prediction probability of the diesel locomotive to be detected under each preset abnormality detection result; and sum the first prediction probability, second prediction probability and third prediction probability of the diesel locomotive to be detected under each preset abnormality detection result respectively to obtain the target prediction probability of the diesel locomotive to be detected under each preset abnormality detection result.

[0160] Each module in the aforementioned diesel locomotive data anomaly detection device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor within a computer device in hardware form, or may be stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0161] In an exemplary embodiment, Figure 6 As shown, a diesel locomotive data anomaly detection system is provided, including: a data acquisition module, a data preprocessing module, a data exploration module and a multidimensional data processing module; the data preprocessing module is communicated with the data acquisition module and the data exploration module respectively, and the data exploration module is communicated with the multidimensional data processing module.

[0162] The data acquisition module is used to obtain the current video data of the diesel locomotive to be detected and send the current video data to the data preprocessing module; the current video data includes the facial video data and driving behavior video data of the driver in the diesel locomotive to be detected, as well as the environmental video data of the diesel locomotive to be detected.

[0163] The data preprocessing module is used to extract the current image features, current video features and current structural features of the current video data respectively, and use the current image features, current video features and current structural features as the current data features of the current video data, and send the current data features to the data exploration module; the current structural features are obtained based on the element features of the target object contained in the current video data.

[0164] The data exploration module is used to determine the abnormal features, trend features and correlation features corresponding to the current data features, and send the abnormal features, trend features and correlation features to the multidimensional data processing module.

[0165] The multi-dimensional data processing module is used to input abnormal features, trend features and correlation features into the trained abnormality detection model to obtain the target abnormality detection results of the diesel locomotive to be detected.

[0166] In an exemplary embodiment, Figure 3 As shown, the multi-dimensional data processing module is also used to send the target anomaly detection results to the data storage module, display module, wireless communication module and warning module respectively.

[0167] The data storage module is used to store the target anomaly detection results.

[0168] The display module is used to determine the driving state corresponding to the target abnormality detection result, and display the driving state when the driving state affects driving safety.

[0169] The wireless communication module is used to wirelessly transmit the target abnormality detection results and remind the crew corresponding to the diesel locomotive to be detected in voice and text form.

[0170] The warning module is used to determine the violation behavior corresponding to the target anomaly detection result, generate alarm information corresponding to the violation behavior, and issue the alarm information.

[0171] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, memory and input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store facial video data, driving behavior video data and environmental video data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for detecting anomalies in diesel locomotive data is implemented.

[0172] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0173] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0174] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0175] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0176] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0177] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0178] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for detecting anomaly in diesel locomotive data, characterized in that: The method comprises: Acquire current video data of the diesel locomotive to be detected; the current video data includes facial video data and driving behavior video data of the driver of the diesel locomotive to be detected, and environmental video data of the diesel locomotive to be detected; extracting current image features, current video features, and current structural features of the current video data respectively, and using the current image features, the current video features, and the current structural features as current data features of the current video data; the current structural features are obtained based on element features of the target object contained in the current video data; Determining abnormal features, trend features, and correlation features corresponding to the current data features; The abnormality feature, the trend feature and the correlation feature are input into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected.

2. The method according to claim 1, characterized in that The extracting the current image features, the current video features and the current structural features of the current video data respectively includes: extracting current image data from the current video data; Performing feature extraction processing on the current image data to obtain image visual features of the current image data; Performing feature extraction again on the visual features of the image to obtain semantic features of the current image data; The image visual features and the semantic features are fused to obtain the current image features.

3. The method according to claim 1, characterized in that The extracting of the current image features, the current video features and the current structural features of the current video data respectively further includes: Performing a time-domain convolution process on the current video data to obtain a time-domain feature of the current video data, and performing a spatial convolution process on the current video data to obtain a spatial feature of the current video data; Fusing the temporal features and the spatial features to obtain local temporal and spatial features of the current video data; The global spatiotemporal features of the current video data are extracted, and the local spatiotemporal features and the global spatiotemporal features are fused to obtain the current video features.

4. The method according to claim 1, wherein The extracting of the current image features, the current video features and the current structural features of the current video data respectively further includes: Performing target detection processing on the current video data to obtain image data of the target object contained in the current video data; Performing feature extraction processing on the image data of the target object to obtain element features of the target object; The element features of each target object are fused to obtain the current structured features.

5. The method according to any one of claims 1 to 4, characterized in that The abnormality feature, the trend feature, and the correlation feature are input into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected, including: Inputting the abnormality feature, the trend feature, and the correlation feature into the trained abnormality detection model to obtain a target prediction probability of the diesel locomotive to be detected under each preset abnormality detection result; From the preset abnormality detection results, the preset abnormality detection results whose target prediction probability is greater than the preset prediction probability are screened out as the target abnormality detection results of the diesel locomotive to be detected.

6. The method according to claim 5, characterized in that The step of inputting the abnormality feature, the trend feature, and the correlation feature into the trained abnormality detection model to obtain a target prediction probability of the diesel locomotive to be detected under each preset abnormality detection result includes: Inputting the abnormality feature into the trained abnormality detection model to obtain a first predicted probability of the diesel locomotive to be detected under each preset abnormality detection result; Inputting the trend feature into the trained anomaly detection model to obtain a second predicted probability of the diesel locomotive to be detected under each preset anomaly detection result; Inputting the associated features into the trained anomaly detection model to obtain a third predicted probability of the diesel locomotive to be detected under each preset anomaly detection result; The first prediction probability, the second prediction probability and the third prediction probability of the diesel locomotive to be detected under each preset abnormal detection result are summed respectively to obtain the target prediction probability of the diesel locomotive to be detected under each preset abnormal detection result.

7. A device for detecting abnormal data of a diesel locomotive, characterized in that: The device comprises: a data acquisition module for acquiring current video data of the diesel locomotive to be detected; the current video data includes facial video data and driving behavior video data of the driver of the diesel locomotive to be detected, and environmental video data of the diesel locomotive to be detected; a feature extraction module, configured to respectively extract current image features, current video features, and current structural features of the current video data, and use the current image features, the current video features, and the current structural features as current data features of the current video data; the current structural features are obtained based on element features of a target object contained in the current video data; A feature determination module, configured to determine abnormal features, trend features, and correlation features corresponding to the current data features; The model detection module is used to input the abnormality feature, the trend feature and the correlation feature into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected.

8. A diesel locomotive data anomaly detection system, characterized in that: The system includes: a data acquisition module, a data preprocessing module, a data exploration module and a multidimensional data processing module; the data preprocessing module is respectively connected to the data acquisition module and the data exploration module, and the data exploration module is connected to the multidimensional data processing module; The data acquisition module is used to obtain current video data of the diesel locomotive to be detected and send the current video data to the data preprocessing module; the current video data includes facial video data and driving behavior video data of the driver of the diesel locomotive to be detected, and environmental video data of the diesel locomotive to be detected; The data preprocessing module is used to extract current image features, current video features, and current structural features of the current video data, respectively, and use the current image features, the current video features, and the current structural features as current data features of the current video data, and send the current data features to the data exploration module; the current structural features are obtained based on the element features of the target object contained in the current video data; The data exploration module is configured to determine abnormal features, trend features, and correlation features corresponding to the current data features, and send the abnormal features, trend features, and correlation features to the multidimensional data processing module; The multi-dimensional data processing module is used to input the abnormality feature, the trend feature and the correlation feature into the trained abnormality detection model to obtain the target abnormality detection result of the diesel locomotive to be detected.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images