Monitoring method and device for special electric vehicle, vehicle-mounted terminal and storage medium

Abstract

The invention relates to the technical field of power systems, in particular to a monitoring method and device for a special power vehicle, a vehicle-mounted terminal and a storage medium, and the method comprises the steps: carrying out the image feature extraction of obtained image monitoring data, and obtaining an image monitoring feature vector; performing data feature extraction on the obtained time sequence monitoring data to obtain a time sequence monitoring feature vector; performing data analysis on the acquired text monitoring data to obtain an exception recognition result; taking the image monitoring feature vector as a Q matrix in a cross attention mechanism, taking the time sequence monitoring feature vector as a K matrix and a V matrix in the cross attention mechanism, and calculating to obtain attention information; inputting the attention information into the monitoring model, executing the monitoring model, and obtaining a part of monitoring results output by the monitoring model; and splicing a part of monitoring results and the anomaly recognition result together to obtain a fusion monitoring result. According to the technical scheme, the special electric vehicle can be comprehensively monitored.

Description

Technical Field

[0001] The present disclosure relates to the technical field of electric power systems, and in particular to a monitoring method, device, vehicle-mounted terminal and storage medium for electric special vehicles. Background Art

[0002] In recent years, the State Grid has continued to strengthen the configuration and use of special electric vehicles to perform daily operations and emergency supply tasks such as non-stop power supply operations, emergency repairs, and power supply for major events. With the increasing demand for special electric vehicles in non-stop power supply maintenance and construction operations, power restoration before repairs, and the frequent occurrence of extreme weather and natural disasters, special electric vehicles are needed to respond quickly to dispatch and command.

[0003] The inventors’ research found that currently special-purpose electric vehicles are constrained by the imperfect digital emergency response system of the distribution network. Basically, they repair the part of the vehicle that has a problem that is detected. The monitoring is not comprehensive and the problem cannot be solved fundamentally. Summary of the invention

[0004] In order to solve the problems in the related art, the embodiments of the present disclosure provide a monitoring method, device, vehicle-mounted terminal and storage medium for a special electric vehicle.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for monitoring an electric special vehicle, comprising: Obtain image monitoring data, time series monitoring data and text monitoring data of electric special vehicles; Performing image feature extraction on the image monitoring data through a preset target detection model to obtain an image monitoring feature vector; Extracting data features from the time series monitoring data using a preset anomaly detection model to obtain a time series monitoring feature vector; Performing data analysis on the text monitoring data using preset data anomaly recognition rules to obtain anomaly recognition results; The image monitoring feature vector is used as the Q matrix in the cross attention mechanism, and the time series monitoring feature vector is used as the K matrix and the V matrix in the cross attention mechanism to calculate the attention information; Inputting the attention information into a preset monitoring model, executing the monitoring model, and obtaining a partial monitoring result output by the monitoring model; The partial monitoring results and the anomaly recognition results are spliced ​​together to obtain a fused monitoring result.

[0006] In a possible implementation, the method further includes: Performing image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result; The anomaly detection model is used to perform anomaly prediction on the time series monitoring feature vector to determine whether the data at the current moment in the time series monitoring data is abnormal.

[0007] In a possible implementation, performing image recognition on the image monitoring feature vector by using the target detection model to obtain a target image recognition result includes: Performing image recognition on the image monitoring data by using a first target detection model to obtain a first image recognition result; Performing image recognition on the image monitoring data by using a second target detection model to obtain a second image recognition result; If the first image recognition result and the second image recognition result are consistent, determining that the target image recognition result is the first image recognition result or the second image recognition result; If the first image recognition result and the second image recognition result are inconsistent, continue to perform image recognition until the first image recognition result and the second image recognition result are consistent, then determine that the target image recognition result is the first image recognition result or the second image recognition result; or until the number of image recognition times exceeds a predetermined number, then randomly select the first image recognition result or the second image recognition result as the target image recognition result.

[0008] In a possible implementation, the extracting image features from the image monitoring data using a preset target detection model to obtain an image monitoring feature vector includes: Obtain an image monitoring feature vector extracted by a target detection model corresponding to the target image recognition result.

[0009] In a possible implementation manner, the acquiring of image monitoring data, time series monitoring data and text monitoring data of electric special vehicles includes: Acquiring image monitoring data of the electric special vehicle through a photographing device; Acquiring time series monitoring data of the electric special vehicle through sensors, wherein the time series monitoring data includes environmental monitoring time series data and index monitoring time series data of predetermined important components; The location information of the special electric vehicle is obtained through a Beidou communication chip, and the vehicle chassis information of the special electric vehicle is obtained through a chassis monitoring device. The text monitoring data includes the location information and the vehicle chassis information.

[0010] In a possible implementation, the method further includes: Inputting the fusion monitoring result into a preset abnormal monitoring decision model, executing the abnormal monitoring decision model, and obtaining a decision output by the abnormal monitoring decision model; The decision is output so that the decision is executed.

[0011] In a second aspect, an embodiment of the present disclosure provides a monitoring device for an electric special vehicle, comprising: A data acquisition module, configured to acquire image monitoring data, time series monitoring data and text monitoring data of electric special vehicles; A first extraction module is configured to extract image features from the image monitoring data using a preset target detection model to obtain an image monitoring feature vector; A second extraction module is configured to extract data features from the time series monitoring data through a preset anomaly detection model to obtain a time series monitoring feature vector; A data analysis module is configured to perform data analysis on the text monitoring data according to a preset data anomaly identification rule to obtain an anomaly identification result; An attention module is configured to use the image monitoring feature vector as a Q matrix in a cross attention mechanism, and use the time series monitoring feature vector as a K matrix and a V matrix in the cross attention mechanism to calculate attention information; A monitoring module, configured to input the attention information into a preset monitoring model, execute the monitoring model, and obtain a partial monitoring result output by the monitoring model; The fusion module is configured to splice the partial monitoring results and the anomaly recognition results together to obtain a fused monitoring result.

[0012] In a possible implementation, the device further includes: An image recognition module is configured to perform image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result; The anomaly prediction module is configured to perform anomaly prediction on the time series monitoring feature vector through the anomaly detection model to determine whether the data at the current moment in the time series monitoring data is abnormal.

[0013] In a possible implementation, the image recognition module is configured as follows: Performing image recognition on the image monitoring data by using a first target detection model to obtain a first image recognition result; Performing image recognition on the image monitoring data by using a second target detection model to obtain a second image recognition result; If the first image recognition result and the second image recognition result are consistent, determining that the target image recognition result is the first image recognition result or the second image recognition result; If the first image recognition result and the second image recognition result are inconsistent, continue to perform image recognition until the first image recognition result and the second image recognition result are consistent, then determine that the target image recognition result is the first image recognition result or the second image recognition result; or until the number of image recognition times exceeds a predetermined number, then randomly select the first image recognition result or the second image recognition result as the target image recognition result.

[0014] In a possible implementation, the first extraction module is configured to: Obtain an image monitoring feature vector extracted by a target detection model corresponding to the target image recognition result.

[0015] In a possible implementation, the data acquisition module is configured as follows: Acquiring image monitoring data of the electric special vehicle through a photographing device; Acquiring time series monitoring data of the electric special vehicle through sensors, wherein the time series monitoring data includes environmental monitoring time series data and index monitoring time series data of predetermined important components; The location information of the special electric vehicle is obtained through a Beidou communication chip, and the vehicle chassis information of the special electric vehicle is obtained through a chassis monitoring device. The text monitoring data includes the location information and the vehicle chassis information.

[0016] In a possible implementation, the device further includes: A decision module is configured to input the fusion monitoring result into a preset abnormal monitoring decision model, execute the abnormal monitoring decision model, and obtain a decision output by the abnormal monitoring decision model; The output module is configured to output the decision so that the decision can be executed.

[0017] In a third aspect, an embodiment of the present disclosure provides a vehicle-mounted terminal, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, and wherein the one or more computer instructions are executed by the processor to implement a method as described in any one of the first aspects.

[0018] In a possible implementation, it further includes: The power distribution security encryption unit is responsible for encrypting and decrypting power distribution business data; Video security encryption unit, responsible for encrypting and decrypting the monitored video data; Beidou communication unit, responsible for the communication of the location information of electric special vehicles and Beidou short message information; The remote communication unit is responsible for the remote communication between the vehicle-mounted terminal and the video monitoring platform and the Internet of Things management platform.

[0019] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, a method as described in any one of the first aspects is implemented.

[0020] According to the technical solution provided by the embodiment of the present disclosure, image monitoring data, time series monitoring data and text monitoring data of electric special vehicles can be obtained; image features of the image monitoring data are extracted through a preset target detection model to obtain an image monitoring feature vector; data features of the time series monitoring data are extracted through a preset anomaly detection model to obtain a time series monitoring feature vector; data analysis is performed on the text monitoring data through a preset data anomaly recognition rule to obtain an anomaly recognition result; the image monitoring feature vector is used as the Q matrix in the cross-attention mechanism, and the time series monitoring feature vector is used as the K matrix and V matrix in the cross-attention mechanism to calculate the attention information; the attention information is input into the monitoring model, and the monitoring model is executed to obtain a partial monitoring result output by the monitoring model; the partial monitoring result and the anomaly recognition result are spliced ​​together to obtain a fused monitoring result; in this way, the three different modalities of image monitoring data, time series monitoring data and text monitoring data of electric special vehicles are integrated and processed to obtain a fused monitoring result, so that the monitoring result is more comprehensive, so as to fundamentally solve the problem based on the comprehensive monitoring result.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings.

[0023] Figure 1 A flow chart of a method for monitoring electric special vehicles provided in an embodiment of the present disclosure is shown.

[0024] Figure 2 A schematic diagram showing the processing flow of the cross-attention mechanism provided by an embodiment of the present disclosure.

[0025] Figure 3 A structural block diagram of a monitoring device for electric special vehicles provided in an embodiment of the present disclosure is shown.

[0026] Figure 4 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0027] Figure 5 A schematic diagram showing the structure of a computer system suitable for implementing the method of the embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0028] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0029] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof exist or are added.

[0030] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0031] Figure 1 FIG. 1 is a flow chart showing a method for monitoring electric special vehicles provided by an embodiment of the present disclosure. Figure 1 As shown, the monitoring method of the electric special vehicle includes the following steps S101-S107: In step S101, image monitoring data, time series monitoring data and text monitoring data of electric special vehicles are obtained; In step S102, image features are extracted from the image monitoring data using a preset target detection model to obtain an image monitoring feature vector; In step S103, data features are extracted from the time series monitoring data using a preset anomaly detection model to obtain a time series monitoring feature vector; In step S104, the text monitoring data is analyzed by a preset data anomaly recognition model to obtain an anomaly recognition result; In step S105, the image monitoring feature vector is used as the Q matrix in the cross attention mechanism, and the time series monitoring feature vector is used as the K matrix and the V matrix in the cross attention mechanism to calculate the attention information; In step S106, the attention information is input into a preset monitoring model, the monitoring model is executed, and a partial monitoring result output by the monitoring model is obtained; In step S107, the partial monitoring results and the abnormality recognition results are spliced ​​together to obtain a fused monitoring result.

[0032] In a possible implementation, the method for monitoring electric special vehicles is applicable to a vehicle-mounted terminal capable of performing monitoring of electric special vehicles.

[0033] In a possible implementation, the following three different modes of monitoring data can be acquired through various monitoring devices mounted on the electric special vehicle: image monitoring data, time series monitoring data, and text monitoring data.

[0034] Among them, the image monitoring data refers to the image data around the special electric vehicle collected by a shooting device such as a camera. The time series monitoring data refers to the time series data related to the operating status of the special electric vehicle detected by the sensor. For example, the time series monitoring data can be environmental series data such as temperature time series data and humidity time series data near the special electric vehicle, or it can be indicator series data of important components on the special electric vehicle. For example, when the special electric vehicle is a power grid emergency power supply vehicle, the time series monitoring data can be indicator series data such as voltage series data and current series data of the generator. The text monitoring data refers to data that can be described in text and monitored by certain specific monitoring equipment, such as the location information of the special electric vehicle, the chassis information of the special electric vehicle, etc.

[0035] In a possible implementation, the monitoring data of the three modes mentioned above can be cross-modally fused to obtain a comprehensive monitoring result, thereby realizing comprehensive monitoring of electric special vehicles. Before fusion, the monitoring data of the three modes can be processed accordingly.

[0036] In a possible implementation, for the image monitoring data, a preset target detection model can be used to extract image features from the image monitoring data to obtain an image monitoring feature vector; the target detection model mentioned here refers to a model that can detect objects in an image, such as an RCNN (Region-based Convolutional Neural Networks) model or a YOLO (You Only LookOnce, unified real-time target detection) model. The input of the target detection model is image data, and the output is an object detected in the image data. The target detection model includes a feature extraction layer and an output layer. The feature extraction layer can extract an image monitoring feature vector from the image monitoring data. After the image monitoring feature vector is input to the output layer, the output layer can output the object detected in the image data. Here, only the image monitoring feature vector is used, and the final output result is not used.

[0037] In a possible implementation, for time series monitoring data, a preset anomaly detection model can be used to extract data features from the time series monitoring data to obtain a time series monitoring feature vector. The anomaly detection model described here can be a model for detecting whether the monitoring data at the current moment is abnormal, for example, it can be an RNN (Recurrent Neural Network) model. The RNN model is a time recursive neural network that is specifically used to process data with sequence or order dependencies. The RNN model will memorize the previous information and apply it to the calculation of the current output, that is, the nodes between the hidden layers are no longer unconnected but connected, and the input of the hidden layer includes not only the output of the input layer but also the output of the hidden layer at the previous moment. The input of the anomaly detection model is the time series monitoring data, and the output is whether the monitoring data at the current moment in the time series monitoring data is abnormal. The time series monitoring data includes a feature extraction layer and an output layer. The feature extraction layer can extract a time series monitoring feature vector from the time series monitoring data. After the time series monitoring feature vector is input into the output layer, the output layer can output whether the monitoring data at the current moment in the time series monitoring data is abnormal. Only the time series monitoring feature vector is used here, and the final output result is not used.

[0038] In a possible implementation, for text monitoring data, the text monitoring data can be analyzed by a preset data anomaly identification rule to obtain an abnormal identification result, and the preset data anomaly identification rule can be a pre-set rule for determining whether the text monitoring data is abnormal. When the text monitoring data is the location information of the electric special vehicle, the preset data anomaly identification rule can be: based on the location information of the electric special vehicle, determine whether the current location of the electric special vehicle is consistent with the location pre-specified by the system. If consistent, the abnormal identification result can be that the electric special vehicle has not arrived at the specified location. If inconsistent, the abnormal identification result can be that the electric special vehicle has arrived at the specified location. When the text monitoring data is vehicle chassis information, the preset data anomaly identification rule can be: compare the current vehicle chassis information with the normal vehicle chassis information. If it does not belong to normal vehicle chassis information, the abnormal identification result is that the vehicle condition is abnormal, and an abnormal alarm can be issued at this time. If it belongs to normal vehicle chassis information, the abnormal identification result is that the vehicle condition is normal.

[0039] In a possible implementation, a cross-attention mechanism can be used to fuse the monitoring data of the two modalities, image monitoring feature vector and time series monitoring feature vector, and analyze to obtain partial monitoring results. In the cross-attention mechanism, an input vector (such as an image monitoring feature vector) can be used as the Q (Query) matrix, and then the attention weights associated with it can be calculated based on another input vector (such as a time series monitoring feature vector) as the K (Key) matrix, and these attention weights are applied to the V (Value) matrix to finally calculate the attention information.

[0040] For example, Figure 2 FIG. 2 is a schematic diagram showing a processing flow of the cross-attention mechanism provided by an embodiment of the present disclosure. Figure 2 As shown, the calculation process of the cross attention mechanism includes: Step 1, using the image monitoring feature vector as the Q matrix in the cross attention mechanism, and the time series monitoring feature vector as the K matrix and V matrix in the cross attention mechanism; Step 2: Calculate the similarity between the Q matrix, i.e., the image monitoring feature vector, and the K matrix, i.e., the time series monitoring feature vector, to calculate the similarity between K and Q, and scale the similarity. The similarity can indicate the attention relationship between the image monitoring feature vector and the time series monitoring feature vector.

[0041] Step 3: Normalize the scaling result with the softmax function to obtain a probability value of 0-1, which represents the attention weight of the query on the key, i.e., the V matrix.

[0042] Step 4: After performing a dot product operation on the probability value, the attention information is obtained, which can be expressed using the following formula: in, is the transpose of Q and K ( ) to perform a dot product operation, indicating and Similarity at different locations; is the dimension of K, which serves as a scaling factor to avoid excessive values; the softmax function converts the scaled similarity Convert to probability distribution; dot product operation between V and probability value is equivalent to extracting the concerned information from value V to obtain attention information .

[0043] In a possible implementation, the attention information is the fused data, which can be input into a monitoring model. The monitoring model makes a comprehensive judgment based on the attention information to obtain partial monitoring results output by the monitoring model. For example, a partial monitoring result may be "a cat broke into the vicinity of the emergency power supply vehicle, which is dangerous. The current output current value of the generator set is 3.3A, the voltage value is 220.12V, and the temperature and humidity of the surrounding environment meet the requirements."

[0044] In a possible implementation, the partial monitoring result obtained in step S106 and the abnormal identification result obtained in step S103 can be spliced ​​together to obtain a fused monitoring result. For example, the partial monitoring result is still as shown in the above example, and the abnormal identification result is "the vehicle is in good condition and has not yet arrived at the designated location." The fused monitoring result obtained by splicing the abnormal identification result and the partial monitoring result together is "a cat broke into the vicinity of the emergency power supply vehicle, which is dangerous. The current output current value of the generator set is: 3.3A, the voltage value is 220.12V, the vehicle is in good condition, the vehicle has not yet arrived at the designated location, and the temperature and humidity of the surrounding environment meet the requirements."

[0045] The present implementation manner can obtain image monitoring data, time series monitoring data and text monitoring data of electric special vehicles; perform image feature extraction on the image monitoring data through a preset target detection model to obtain an image monitoring feature vector; perform data feature extraction on the time series monitoring data through a preset anomaly detection model to obtain a time series monitoring feature vector; perform data analysis on the text monitoring data through a preset data anomaly recognition rule to obtain an anomaly recognition result; use the image monitoring feature vector as the Q matrix in the cross-attention mechanism, and use the time series monitoring feature vector as the K matrix and V matrix in the cross-attention mechanism to calculate the attention information; input the attention information into the monitoring model, execute the monitoring model, and obtain partial monitoring results output by the monitoring model; splice the partial monitoring results and the anomaly recognition results together to obtain a fused monitoring result; in this way, the three different modalities of image monitoring data, time series monitoring data and text monitoring data of electric special vehicles are integrated and processed to obtain a fused monitoring result, so that the monitoring result is more comprehensive, so as to fundamentally solve the problem based on the comprehensive monitoring result.

[0046] In a possible implementation, the method further includes: Performing image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result; The anomaly detection model is used to perform anomaly prediction on the time series monitoring feature vector to determine whether the data at the current moment in the time series monitoring data is abnormal.

[0047] In this embodiment, in addition to the above-mentioned fusion monitoring results, the image monitoring feature vector can also be used for image recognition through the target detection model to obtain the target image recognition result, and the time series monitoring feature vector can be used for anomaly prediction through the anomaly detection model to determine whether the data at the current moment in the time series monitoring data is abnormal.

[0048] In a possible implementation, performing image recognition on the image monitoring feature vector by using the target detection model to obtain a target image recognition result includes: Performing image recognition on the image monitoring data by using a first target detection model to obtain a first image recognition result; Performing image recognition on the image monitoring data by using a second target detection model to obtain a second image recognition result; If the first image recognition result and the second image recognition result are consistent, determining that the target image recognition result is the first image recognition result or the second image recognition result; If the first image recognition result and the second image recognition result are inconsistent, continue to perform image recognition until the first image recognition result and the second image recognition result are consistent, then determine that the target image recognition result is the first image recognition result or the second image recognition result; or until the number of image recognition times exceeds a predetermined number, then randomly select the first image recognition result or the second image recognition result as the target image recognition result.

[0049] In this embodiment, the first target detection model can be an RCNN model, and the second target detection model can be a YOLO model. The YOLO model and the RCNN model are two different target detection models. The RCNN model uses a two-step detection method to pre-generate some candidate boxes, and then classify the content in the candidate boxes and correct the positions of the candidate boxes. The YOLO model uses a one-step detection method to directly predict and classify candidate boxes at various positions of the image without pre-generating some candidate boxes. It should be noted here that the first target detection model and the second target detection model can also be other types of target monitoring models, such as the Cascade R-CNN model, the DETR (Detection Transformer) model, and the like.

[0050] In this embodiment, target detection can be performed on the image monitoring data based on the YOLO model and RCNN respectively. If the image recognition results detected by the two models are the same object, the result is considered correct. For example, the camera detects that a cat has entered the danger zone around the emergency power supply vehicle. The cat is identified by the RCNN model and the YOLO model respectively. If the image recognition results given are all cats, the image recognition result obtained is that the cat has entered the danger zone around the emergency power supply vehicle. If the results of the two models are inconsistent, image recognition is performed again. This cycle is repeated for a predetermined number of times, such as 50 times. If the results are still inconsistent, any one of them is selected as the image recognition result.

[0051] In a possible implementation, the extracting image features from the image monitoring data using a preset target detection model to obtain an image monitoring feature vector includes: Obtain an image monitoring feature vector extracted by a target detection model corresponding to the target image recognition result.

[0052] In this embodiment, when the above two target detection models respectively perform image recognition on the image monitoring data, they will first extract the corresponding image monitoring feature vector from the image monitoring data, and then perform image recognition based on the extracted image monitoring feature vector. If the first image recognition result recognized by a certain target detection model such as the first target detection model is finally obtained as the image recognition result, then the first image monitoring feature vector extracted by the first target detection model can be obtained as the image monitoring feature vector.

[0053] In a possible implementation manner, the acquiring of image monitoring data, time series monitoring data and text monitoring data of electric special vehicles includes: Acquiring image monitoring data of the electric special vehicle through a photographing device; Acquiring time series monitoring data of the electric special vehicle through sensors, wherein the time series monitoring data includes environmental monitoring time series data and index monitoring time series data of predetermined important components; The location information of the special electric vehicle is obtained through a Beidou communication chip, and the vehicle chassis information of the special electric vehicle is obtained through a chassis monitoring device. The text monitoring data includes the location information and the vehicle chassis information.

[0054] In this embodiment, the shooting device can be an image sensor such as a camera, which can be installed on the special electric vehicle to capture images around the special electric vehicle and send the captured images to the vehicle-mounted terminal via a PoE (Power over Ethernet) interface. In this way, the vehicle-mounted terminal can obtain image monitoring data of the special electric vehicle.

[0055] In this embodiment, the environmental monitoring time series data can be time series data of at least one environmental parameter such as temperature time series, humidity time series and smoke concentration time series, wherein the temperature time series can be collected by a temperature sensor, the humidity time series can be collected by a humidity sensor, and the smoke concentration time series can be collected by a smoke sensor. These sensors can be connected to the vehicle-mounted terminal for communication via an RS485-1 interface (a standard serial communication interface). In this way, the vehicle-mounted terminal can obtain the environmental monitoring time series data through the RS485-1 interface.

[0056] In this embodiment, different types of electric special vehicles may have different predetermined important components. For example, when the electric special vehicle is a power grid emergency power supply vehicle, the predetermined important component may be a generator set. When the electric special vehicle is a mobile energy storage vehicle, the predetermined important component may be an energy storage component, etc. For the predetermined important component, the corresponding index of the predetermined important component may be monitored to obtain the index monitoring time series data of the predetermined important component. For example, if the predetermined important component is a generator set, its corresponding index may be voltage, current, power, frequency, power generation, circuit breaker opening and closing position, oil pressure, coolant temperature, working time, battery voltage and other generator set index information. By monitoring these indexes, the corresponding index monitoring time series data may be obtained. The generator set may be connected to the generator set and its controller via the RS485-3 interface. In this way, the vehicle-mounted terminal may obtain the index monitoring time series data of the generator set via the RS485-3 interface.

[0057] In this implementation, the Beidou communication chip can be set in the vehicle-mounted terminal, and the vehicle-mounted terminal can obtain the location information of the special electric vehicle through the Beidou communication chip, and the location information includes: longitude, latitude and altitude; in addition to the communication of load location information, the Beidou communication chip is also responsible for the communication of Beidou short message information, etc. The Beidou short message information includes the uplink of key information under 4G / 5G abnormal conditions. In this way, on the basis of general 4G / 5G, the functions of Beidou positioning and Beidou short message are added for mobile operation scenarios, thereby improving the terminal's full-scene communication capability, improving the unified dispatching and command capability, and improving the safety and reliability of the vehicle.

[0058] In this killing mode, the chassis monitoring device can be set in the vehicle chassis, and the chassis monitoring device can monitor the vehicle chassis information, which includes: engine speed, oil level, vehicle speed, fault code and other information. The vehicle chassis can be connected to the vehicle terminal through a CAN (Controller Area Network) bus, so that the vehicle terminal can obtain the vehicle chassis information monitored by the chassis monitoring device through the CAN bus.

[0059] In a possible implementation, the method further includes: Inputting the fusion monitoring result into a preset abnormal monitoring decision model, executing the abnormal monitoring decision model, and obtaining a decision output by the abnormal monitoring decision model; The decision is output so that the decision is executed.

[0060] In this implementation, the abnormal monitoring decision model can be pre-trained by other devices and then set in the vehicle terminal, which is used to analyze the fusion monitoring results to obtain the corresponding decision to respond to the fusion monitoring results. The sample data used for training can be the fusion monitoring results and historical correct decisions in the historical time period. The vehicle terminal can use the fusion monitoring results as the input of the abnormal monitoring decision model, and finally the abnormal monitoring decision model generates and outputs the decision. In this way, the vehicle terminal can perform local processing as a core node, and can make local decisions in time in an emergency, meeting the efficient command requirements of emergency supply support tasks.

[0061] In this implementation, the vehicle-mounted terminal can output the decision through voice, text, pictures, etc., and notify local staff to execute the decision in a timely manner.

[0062] The present disclosure also provides a monitoring device for electric special vehicles, Figure 3 The structure block diagram of the monitoring device for electric special vehicles provided by the embodiment of the present disclosure is shown. The device can be implemented as part or all of the electronic equipment through software, hardware or a combination of both. Figure 3 As shown, the monitoring device for the electric special vehicle includes: The data acquisition module 301 is configured to acquire image monitoring data, time series monitoring data and text monitoring data of the electric special vehicle; The first extraction module 302 is configured to extract image features from the image monitoring data using a preset target detection model to obtain an image monitoring feature vector; The second extraction module 303 is configured to extract data features from the time series monitoring data through a preset anomaly detection model to obtain a time series monitoring feature vector; The data analysis module 304 is configured to perform data analysis on the text monitoring data according to a preset data anomaly identification rule to obtain an anomaly identification result; The attention module 305 is configured to use the image monitoring feature vector as a Q matrix in a cross attention mechanism, and use the time series monitoring feature vector as a K matrix and a V matrix in a cross attention mechanism to calculate attention information; A monitoring module 306 is configured to input the attention information into a monitoring model, execute the monitoring model, and obtain a partial monitoring result output by the monitoring model; The fusion module 307 is configured to combine the partial monitoring results and the abnormality recognition results to obtain a fused monitoring result.

[0063] In a possible implementation, the device further includes: An image recognition module is configured to perform image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result; The anomaly prediction module is configured to perform anomaly prediction on the time series monitoring feature vector through the anomaly detection model to determine whether the data at the current moment in the time series monitoring data is abnormal.

[0064] In a possible implementation, the image recognition module is configured as follows: Performing image recognition on the image monitoring data by using a first target detection model to obtain a first image recognition result; Performing image recognition on the image monitoring data by using a second target detection model to obtain a second image recognition result; If the first image recognition result and the second image recognition result are consistent, determining that the target image recognition result is the first image recognition result or the second image recognition result; If the first image recognition result and the second image recognition result are inconsistent, continue to perform image recognition until the first image recognition result and the second image recognition result are consistent, then determine that the target image recognition result is the first image recognition result or the second image recognition result; or until the number of image recognition times exceeds a predetermined number, then randomly select the first image recognition result or the second image recognition result as the target image recognition result.

[0065] In a possible implementation, the first extraction module is configured to: Obtain an image monitoring feature vector extracted by a target detection model corresponding to the target image recognition result.

[0066] In a possible implementation, the data acquisition module is configured as follows: Acquiring image monitoring data of the electric special vehicle through a photographing device; Acquiring time series monitoring data of the electric special vehicle through sensors, wherein the time series monitoring data includes environmental monitoring time series data and index monitoring time series data of predetermined important components; The location information of the special electric vehicle is obtained through a Beidou communication chip, and the vehicle chassis information of the special electric vehicle is obtained through a chassis monitoring device. The text monitoring data includes the location information and the vehicle chassis information.

[0067] In a possible implementation, the device further includes: A decision module is configured to input the fusion monitoring result into a preset abnormal monitoring decision model, execute the abnormal monitoring decision model, and obtain a decision output by the abnormal monitoring decision model; The output module is configured to output the decision so that the decision can be executed.

[0068] The technical terms and technical features mentioned in the implementation manner of the present device are the same as or similar to those mentioned in the implementation manner of the above method. For the explanation and description of the technical terms and technical features involved in the present device, reference may be made to the explanation and description of the implementation manner of the above method, and they will not be repeated here.

[0069] The present disclosure also discloses an electronic device, Figure 4 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0070] like Figure 4 As shown, the electronic device 400 includes a memory 401 and a processor 402, wherein the memory 401 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 402 to implement the method according to an embodiment of the present disclosure.

[0071] In this implementation, the vehicle-mounted terminal can obtain image monitoring data, time series monitoring data, text monitoring data and other data provided by various monitoring devices on the special electric vehicle. The vehicle-mounted terminal, as a core node, can integrate and process these data to obtain integrated monitoring results. If an emergency occurs and there is no time to communicate with the platform, the vehicle-mounted terminal can directly make local decisions, thereby giving full play to the core functions of the edge computing node.

[0072] In a possible implementation, the vehicle-mounted terminal further includes: The power distribution security encryption unit is responsible for encrypting and decrypting power distribution business data; Video security encryption unit, responsible for encrypting and decrypting the monitored video data; Beidou communication unit, responsible for the communication of the location information of electric special vehicles and Beidou short message information; The remote communication unit is responsible for the remote communication between the vehicle-mounted terminal and the video monitoring platform and the Internet of Things management platform.

[0073] In this embodiment, the vehicle terminal can obtain the monitored video data through the shooting device, encrypt the video data using the video security encryption unit, and send it to the video monitoring platform through the remote communication unit to ensure the security of the video data. The video monitoring platform can use the video data returned by each vehicle terminal to monitor abnormal videos, and send alarm information to the corresponding vehicle equipment in a timely manner when abnormal videos are monitored.

[0074] In this embodiment, the vehicle-mounted terminal can also transmit and interact with the power distribution business data through the remote communication unit and the Internet of Things management platform. When transmitting the power distribution business data, the power distribution security encryption unit can be used for encryption and decryption to ensure the data security of the power distribution business data.

[0075] In this implementation, the Beidou communication unit can communicate with satellites to obtain the location information of special electric vehicles, and can also be responsible for the communication of Beidou short message information in the case of 4G / 5G abnormalities, and the Beidou short message information includes key information uplink in the case of 4G / 5G abnormalities. In this way, on the basis of general 4G / 5G, Beidou positioning and Beidou short messages are added to improve the terminal's full-scenario communication capabilities, improve unified dispatching and command capabilities, and improve the safety and reliability of emergency power supply vehicles.

[0076] This implementation method collects relevant monitoring data of the special electric vehicle by adding on-board terminals, photographing devices, environmental monitoring sensors and other monitoring devices to the special electric vehicle, which can sense and upload key operation data such as vehicle location, operation and maintenance personnel, and working status. The remote communication unit of the on-board terminal can interact with the command system in real time to meet the needs of online real-time monitoring of the status of the special electric vehicle, emergency repair and power restoration progress and other information, comprehensively improve the emergency support perception and auxiliary decision-making capabilities of pre-warning, in-process monitoring, and post-event review, and meet the needs of efficient command of emergency supply support tasks.

[0077] Figure 5 A schematic diagram showing the structure of a computer system suitable for implementing the method of the embodiment of the present disclosure is shown.

[0078] like Figure 5 As shown, the computer system 500 includes a processing unit 501, which can perform various processes in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the computer system 500 are also stored. The processing unit 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0079] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed. Among them, the processing unit 501 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0080] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes computer instructions, and the computer instructions are executed by a processor to implement the method steps described above. In such an embodiment, the computer program product can be downloaded and installed from a network through the communication part 509, and / or installed from a removable medium 511.

[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0082] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or programmable hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0083] As another aspect, the present disclosure further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores one or more programs, and the programs are used by one or more processors to execute the method described in the present disclosure.

[0084] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. A monitoring method for electric special vehicles, characterized in that: include: Obtain image monitoring data, time series monitoring data and text monitoring data of electric special vehicles; Performing image feature extraction on the image monitoring data through a preset target detection model to obtain an image monitoring feature vector; Extracting data features from the time series monitoring data using a preset anomaly detection model to obtain a time series monitoring feature vector; Performing data analysis on the text monitoring data using preset data anomaly recognition rules to obtain anomaly recognition results; The image monitoring feature vector is used as the Q matrix in the cross attention mechanism, and the time series monitoring feature vector is used as the K matrix and the V matrix in the cross attention mechanism to calculate the attention information; Inputting the attention information into a preset monitoring model, executing the monitoring model, and obtaining a partial monitoring result output by the monitoring model; The partial monitoring results and the anomaly recognition results are spliced ​​together to obtain a fused monitoring result.

2. The method according to claim 1, characterized in that The method further comprises: Performing image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result; The anomaly detection model is used to perform anomaly prediction on the time series monitoring feature vector to determine whether the data at the current moment in the time series monitoring data is abnormal.

3. The method according to claim 2, characterized in that The performing image recognition on the image monitoring feature vector by the target detection model to obtain a target image recognition result includes: Performing image recognition on the image monitoring data by using a first target detection model to obtain a first image recognition result; Performing image recognition on the image monitoring data by using a second target detection model to obtain a second image recognition result; If the first image recognition result and the second image recognition result are consistent, determining that the target image recognition result is the first image recognition result or the second image recognition result; If the first image recognition result and the second image recognition result are inconsistent, continue to perform image recognition until the first image recognition result and the second image recognition result are consistent, then determine that the target image recognition result is the first image recognition result or the second image recognition result; or until the number of image recognition times exceeds a predetermined number, then randomly select the first image recognition result or the second image recognition result as the target image recognition result.

4. The method according to claim 3, characterized in that The extracting image features from the image monitoring data by using a preset target detection model to obtain an image monitoring feature vector includes: Obtain an image monitoring feature vector extracted by a target detection model corresponding to the target image recognition result.

5. The method according to claim 1, characterized in that The acquisition of image monitoring data, time series monitoring data and text monitoring data of electric special vehicles includes: Acquiring image monitoring data of the electric special vehicle through a photographing device; Acquiring time series monitoring data of the electric special vehicle through sensors, wherein the time series monitoring data includes environmental monitoring time series data and index monitoring time series data of predetermined important components; The location information of the special electric vehicle is obtained through a Beidou communication chip, and the vehicle chassis information of the special electric vehicle is obtained through a chassis monitoring device. The text monitoring data includes the location information and the vehicle chassis information.

6. The method according to claim 1, characterized in that The method further comprises: Inputting the fusion monitoring result into a preset abnormal monitoring decision model, executing the abnormal monitoring decision model, and obtaining a decision output by the abnormal monitoring decision model; The decision is output so that the decision is executed.

7. A monitoring device for electric special vehicles, characterized in that: include: A data acquisition module, configured to acquire image monitoring data, time series monitoring data and text monitoring data of electric special vehicles; A first extraction module is configured to extract image features from the image monitoring data using a preset target detection model to obtain an image monitoring feature vector; A second extraction module is configured to extract data features from the time series monitoring data through a preset anomaly detection model to obtain a time series monitoring feature vector; A data analysis module is configured to perform data analysis on the text monitoring data according to a preset data anomaly identification rule to obtain an anomaly identification result; An attention module is configured to use the image monitoring feature vector as a Q matrix in a cross attention mechanism, and use the time series monitoring feature vector as a K matrix and a V matrix in the cross attention mechanism to calculate attention information; A monitoring module, configured to input the attention information into a preset monitoring model, execute the monitoring model, and obtain a partial monitoring result output by the monitoring model; The fusion module is configured to splice the partial monitoring results and the anomaly recognition results together to obtain a fused monitoring result.

8. The device according to claim 7, characterized in that The device also includes: An image recognition module is configured to perform image recognition on the image monitoring feature vector through the target detection model to obtain a target image recognition result; The anomaly prediction module is configured to perform anomaly prediction on the time series monitoring feature vector through the anomaly detection model to determine whether the data at the current moment in the time series monitoring data is abnormal.

9. The device according to claim 8, characterized in that The image recognition module is configured as follows: Performing image recognition on the image monitoring data by using a first target detection model to obtain a first image recognition result; Performing image recognition on the image monitoring data by using a second target detection model to obtain a second image recognition result; If the first image recognition result and the second image recognition result are consistent, determining that the target image recognition result is the first image recognition result or the second image recognition result; If the first image recognition result and the second image recognition result are inconsistent, continue to perform image recognition until the first image recognition result and the second image recognition result are consistent, then determine that the target image recognition result is the first image recognition result or the second image recognition result; or until the number of image recognition times exceeds a predetermined number, then randomly select the first image recognition result or the second image recognition result as the target image recognition result.

10. The device according to claim 9, characterized in that The first extraction module is configured to: Obtain an image monitoring feature vector extracted by a target detection model corresponding to the target image recognition result.

11. The device according to claim 7, characterized in that The data acquisition module is configured as follows: Acquiring image monitoring data of the electric special vehicle through a photographing device; Acquiring time series monitoring data of the electric special vehicle through sensors, wherein the time series monitoring data includes environmental monitoring time series data and index monitoring time series data of predetermined important components; The location information of the special electric vehicle is obtained through a Beidou communication chip, and the vehicle chassis information of the special electric vehicle is obtained through a chassis monitoring device. The text monitoring data includes the location information and the vehicle chassis information.

12. The device according to claim 7, characterized in that The device also includes: A decision module is configured to input the fusion monitoring result into a preset abnormal monitoring decision model, execute the abnormal monitoring decision model, and obtain a decision output by the abnormal monitoring decision model; The output module is configured to output the decision so that the decision can be executed.

13. A vehicle-mounted terminal, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to any one of claims 1 to 6.

14. The vehicle-mounted terminal according to claim 13, characterized in that: Also includes: The power distribution security encryption unit is responsible for encrypting and decrypting power distribution business data; Video security encryption unit, responsible for encrypting and decrypting the monitored video data; Beidou communication unit, responsible for the communication of the location information of electric special vehicles and Beidou short message information; The remote communication unit is responsible for the remote communication between the vehicle-mounted terminal and the video monitoring platform and the Internet of Things management platform.

15. A readable storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed by a processor, the method described in any one of claims 1 to 6 is implemented.

Images