A method for transmitting data fusion features of heterogeneous devices in Internet of Vehicles to the cloud

Abstract

The present invention provides a method for transmitting data fusion features of heterogeneous devices in an Internet of Vehicles to the cloud. By combining a multi-source heterogeneous data fusion algorithm, feature extraction and unsupervised fusion processing are performed on heterogeneous data such as text data, vibration data and audio data, and the fusion is reduced to fusion feature data with low data volume for transmission to the cloud; the fusion feature data of the vehicle-mounted edge device is analyzed and converted in the cloud, restored to the preset text data, vibration data and audio data format, and the working data of the vehicle-mounted edge device is stored, and corresponding device identifiers and corresponding data models are respectively set for different vehicle-mounted edge devices. The method of the present invention can reduce the amount of data transmitted when the heterogeneous data of the vehicle-mounted edge device is transmitted to the cloud, save the network bandwidth for transmission to the cloud, reduce its network load, help improve the data interaction efficiency within the Internet of Vehicles system, and enhance the overall perception ability of the Internet of Vehicles system to vehicle status information.

Description

Technical Field

[0001] The present invention relates to the field of data input and processing of heterogeneous devices in an Internet of Things environment, and in particular to a method for transmitting data fusion features of heterogeneous devices in an Internet of Vehicles to the cloud. Background Art

[0002] With the continuous advancement of science and technology, the Internet of Things (IoT) has become an important bridge connecting the world, connecting various devices and systems intelligently through the Internet. In this digital age, connected vehicles (Connected Vehicles), as one of the important application areas of the Internet of Things, is gaining increasing attention and importance. Connected Vehicles technology connects cars to the Internet, which not only makes driving more intelligent and convenient, but also provides safer, more efficient and intelligent transportation solutions. In the development of connected vehicles, the combination of various sensors, communication technologies and data analysis technologies has provided vehicles with more intelligent functions, thereby promoting the upgrade and transformation of the entire transportation system.

[0003] In the field of Internet of Vehicles, the data generated by vehicles includes not only traditional on-board sensor data, but also multi-source heterogeneous data from multiple sources, such as audio, vibration, and text data. These data may come from different devices on the vehicle, which can be called on-board edge devices. The comprehensive use of data related to these on-board edge devices is crucial to achieving a smarter and safer transportation system. Audio data can be used to identify the ambient sound around the vehicle, vibration data can help monitor the status and health of the vehicle, and text data can contain information from on-board systems, navigation applications, and communication devices. By effectively integrating and analyzing these multi-source heterogeneous data, we can achieve more accurate vehicle health monitoring, smarter driver assistance systems, and more efficient traffic management. This comprehensive use of different types of data will bring more possibilities to the development of Internet of Vehicles technology and promote the development of intelligent transportation systems in a more intelligent and sustainable direction.

[0004] As the scale of the Internet of Vehicles expands and the types of vehicle sensors increase, the amount of data collected is also growing exponentially, and is expected to exceed 40Gbit per second. If this amount of data is transmitted directly through the communication system, it will cause serious congestion, and heterogeneous device data will be difficult to manage efficiently. The Internet of Vehicles needs a more efficient system and method for transmitting heterogeneous device data or fusion features to the cloud.

[0005] At present, vehicle sensors mainly work independently, such as using cameras for image recognition and radars for speed and distance detection. Vehicles still lack the ability to effectively integrate multiple sensor data. The Internet of Vehicles needs to integrate multi-source heterogeneous data to reduce the amount of transmitted data, save network bandwidth, reduce network load, and improve the overall perception ability of the vehicle.

[0006] Traditional centralized computing and processing methods can no longer meet the needs of the Internet of Vehicles. Although the centralized processing and computing capabilities of the cloud are powerful, their communication transmission performance does not match the computing performance, which easily leads to data congestion problems. In order to meet these challenges, the Internet of Vehicles needs to shift to a more distributed computing mode, using edge computing to preprocess data, reduce data transmission volume, and improve real-time performance and efficiency. However, compared with the growth rate of sensor data volume, the development of edge acquisition computing devices is relatively slow, and the edge acquisition computing devices of a single vehicle are difficult to cope with the huge challenges of data fusion processing. In order to solve this problem, the Internet of Vehicles needs to share data and resources among the computing resources of equipment such as vehicle terminals, roadside units, and base stations. This collaborative computing mode can effectively meet the needs of massive data processing. By allocating data processing tasks to multiple edge acquisition computing devices, efficient data processing and analysis can be achieved, thereby ensuring that the Internet of Vehicles system can operate efficiently and provide more intelligent traffic solutions. Summary of the invention

[0007] In view of the deficiencies of the above-mentioned prior art, the present invention provides a method for transmitting the data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud, so as to reduce the amount of data transmitted to the cloud for heterogeneous data of vehicle-mounted edge devices, save network bandwidth for transmission to the cloud, reduce its network load, and thereby help improve the data interaction efficiency within the Internet of Vehicles system and enhance the overall perception capability of the Internet of Vehicles system of vehicle status information.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] A method for transmitting data fusion features of heterogeneous devices in an Internet of Vehicles to the cloud comprises the following steps:

[0010] S1. Construct the device identification and data model corresponding to each vehicle-mounted edge device in the Internet of Things in the cloud; the device model corresponding to the vehicle-mounted edge device is used to identify the individual identity of the vehicle-mounted edge device; the data model corresponding to the vehicle-mounted edge device is used to record the working data of the corresponding vehicle-mounted edge device;

[0011] S2. Deploy edge collection and computing devices in the preset data collection area. Each edge collection and computing device can establish a data communication connection with multiple vehicle-mounted edge devices, and build a data transmission channel between the edge collection and computing device and the cloud, so that the working data collected by the vehicle-mounted edge device can be transmitted to the cloud through the edge collection and computing device;

[0012] S3, the vehicle-mounted edge device collects its working data and transmits it to the edge collection and computing device with which it establishes a data communication connection; the working data includes text data, vibration data and audio data of the vehicle-mounted edge device;

[0013] S4. The edge acquisition computing device performs feature extraction and unsupervised fusion processing on the text data, vibration data and audio data in the working data of the vehicle-mounted edge device to obtain the fused feature data of the vehicle-mounted edge device, and uploads it to the cloud through the data transmission channel;

[0014] S5. The cloud side parses and converts the fused feature data of the vehicle-mounted edge device, and obtains the text recovery data, vibration recovery data, and audio recovery data of the vehicle-mounted edge device, which are stored as working parsed data in the data model corresponding to the vehicle-mounted edge device.

[0015] As a preferred solution, in step S1, a data storage space for the vehicle-mounted edge device and its components is set in the data model corresponding to the vehicle-mounted edge device, which is used to store the working data of the vehicle-mounted edge device and the working data of its components respectively.

[0016] As a preferred solution, step S2 includes:

[0017] S201, establishing a device access gateway between the cloud and the edge collection and computing device, each device access gateway is used as a data access point for one or more edge collection and computing devices;

[0018] S202. Establish a data access channel in the cloud to serve as a data transmission channel between the cloud and the device access gateway, so that the working data of the vehicle-mounted edge device collected by the edge collection and computing device can be uploaded to the cloud through the device access gateway and the data access channel.

[0019] As a preferred solution, step S4 includes:

[0020] S401, the vehicle-mounted edge device communicates data with the edge collection and computing device, and the edge collection and computing device collects working data of the vehicle-mounted edge device;

[0021] S402, the edge collection and computing device performs preprocessing of word segmentation, sentence segmentation, and stop word removal on the text data in the working data;

[0022] S403, the edge collection and calculation device performs word embedding feature extraction on the preprocessed text data to obtain a word feature vector of the text data;

[0023] S404, the edge acquisition and calculation device performs slicing and segmenting processing on the vibration data and the audio data in the working data in time sequence to obtain a vibration data slice vector and an audio data slice vector;

[0024] S405. The edge acquisition and computing device performs unsupervised dimensionality reduction and fusion processing on the word feature vectors of the text data, the vibration data slice vectors and the audio data slice vectors to obtain fused feature data of the vehicle-mounted edge device.

[0025] As a preferred solution, in step S401, the vehicle-mounted edge device establishes a data connection with an edge acquisition and computing device that can establish data communication, obtains the data processing resource occupancy rate of the edge acquisition and computing device, and selects an edge acquisition and computing device whose data processing resource occupancy rate is less than a preset resource occupancy threshold to upload the working data of the vehicle-mounted edge device.

[0026] As a preferred solution, in step S402, the edge acquisition computing device uses a Bert model that has been pre-trained for text preprocessing to perform word segmentation, sentence segmentation, and stop word removal on the text data in the working data.

[0027] As a preferred solution, in step S403, the edge acquisition computing device uses a pre-trained Bert-base-chinese model to extract word embedding features for Chinese text data, and uses a pre-trained Bert-base-uncased model to extract word embedding features for English text data; the word feature vector representation of the obtained text data is dimensional matrix vector, a single sentence in the text data is represented as n i ×N x dimensional vector, a single character is represented as 1×N x dimensional vector, N x Indicates the feature dimension of feature extraction, n i represents the number of characters in the i-th sentence in the text data, and m represents the number of sentences obtained after the text data is segmented, so that the word feature vector not only contains the character information and semantic information of the global text in the text data, but also contains the position information of each sentence and character.

[0028] As a preferred solution, in step S404, the edge acquisition computing device reads the vibration data and audio data in the working data respectively and performs normalization processing, and then extracts the feature dimension N according to the preset feature. x , the vibration data and audio data are sliced ​​and segmented; the obtained vibration data slice vector is represented as x×Nx dimensional vector, the audio data slice vector is represented as y×N x dimensional vectors, x and y represent the number of slice segments of the vibration data and the audio data respectively, so that the vibration data slice vector and the audio data slice vector not only contain the amplitude characteristic information of the vibration data and the audio data respectively, but also contain the position information of each slice segment.

[0029] As a preferred solution, in step S405, the edge acquisition computing device first performs splicing and fusion processing on the word feature vector of the text data, the vibration data slice vector and the audio data slice vector to obtain a working data feature matrix vector, and then inputs the working data feature matrix vector into a pre-trained Autoencoder self-encoding model for unsupervised dimensionality reduction encoding processing to obtain fused feature data of the vehicle-mounted edge device; wherein the working data feature matrix vector is represented as dimensional matrix vector, N x Indicates the feature dimension of feature extraction, n i represents the number of characters in the i-th sentence in the text data, m represents the number of sentences obtained after the text data is segmented, and x and y represent the number of slice segmentations of the vibration data and the audio data, respectively.

[0030] As a preferred solution, in step S5, the cloud first decodes the fused feature data of the vehicle-mounted edge device through the Autoencoder self-encoding process to obtain The working data feature matrix vector of the vehicle-mounted edge device is then decomposed to obtain The word feature vector of the text data of x×N x dimensional vibration data slice vector and y×N x dimensional audio data slice vector, and according to the position information contained in the word feature vector of the text data, the vibration data slice vector and the audio data slice vector, feature reorganization is performed on the word features of the text data and the slice segments of the vibration data and the audio data in position order, and then the respective reorganization results are converted into data formats according to the preset text data format, vibration data format and audio data format, to obtain the text recovery data, vibration recovery data and audio recovery data of the vehicle-mounted edge device, and finally the text recovery data, vibration recovery data and audio recovery data of the vehicle-mounted edge device are used as the working parsing data of the corresponding vehicle-mounted edge device, and stored in the data model corresponding to the corresponding vehicle-mounted edge device.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The present invention proposes a method for transmitting the data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud. In view of the large amount of heterogeneous data of vehicle-mounted edge devices, which easily causes communication congestion in cloud transmission, the method combines a multi-source heterogeneous data fusion algorithm to extract features and perform unsupervised fusion processing on heterogeneous data such as text data, vibration data and audio data, and then reduces the dimensionality of the fused feature data with low data volume for transmission to the cloud, thereby saving network bandwidth for cloud transmission and reducing its network load.

[0033] 2. In the method for transmitting the fusion feature data of heterogeneous devices in the Internet of Vehicles to the cloud of the present invention, the fusion feature data of the vehicle-mounted edge device is analyzed and converted in the cloud and restored to the preset text data, vibration data and audio data format, and the working data of the vehicle-mounted edge device is stored. In addition, corresponding device identifiers and corresponding data models are set for different vehicle-mounted edge devices, which facilitates the differentiation of vehicle-mounted edge devices for the stored heterogeneous data such as text, audio, vibration, etc., and provides data support for subsequent equipment operation and maintenance and other applications.

[0034] 3. The method of the present invention can reduce the amount of heterogeneous data of vehicle-mounted edge devices when transmitting them to the cloud, save network bandwidth for cloud transmission, and reduce its network load. At the same time, it can also well ensure the storage integrity of future data of vehicle-mounted edge devices in the cloud, thereby helping to improve the data interaction efficiency within the Internet of Vehicles system and enhance the overall perception capability of the Internet of Vehicles system of vehicle status information, and has great value for technology promotion and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0036] Figure 1 It is a flow chart of the method for transmitting data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud according to the present invention.

[0037] Figure 2 It is an overall working diagram of an application example of the method for transmitting data fusion features of heterogeneous devices in Internet of Vehicles to the cloud of the present invention.

[0038] Figure 3 Schematic diagram of a device model constructed in an embodiment of the present invention.

[0039] Figure 4 It is a schematic diagram of the principle of the feature fusion algorithm of audio, text and vibration data in an embodiment of the present invention.

[0040] Figure 5 It is a schematic diagram of the principle of extracting sentence data by the Bert model in an embodiment of the present invention.

[0041] Figure 6It is a structural schematic diagram of the AE model in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0043] In view of the deficiencies in the prior art, the present invention mainly aims to solve the following problems:

[0044] (1) With the development of the Internet of Things and sensor technology, the amount of collected data is growing exponentially. Direct transmission through the communication system is likely to cause cloud transmission communication congestion. In addition, heterogeneous device data is difficult to manage efficiently and is difficult to operate manually.

[0045] (2) Due to the large amount of data and the non-uniform data structure, the data from multi-source heterogeneous devices is not fully utilized, resulting in poor overall perception of the vehicle-mounted devices by the Internet of Vehicles system.

[0046] (3) The processing and computing power of the on-board edge equipment is insufficient and it is difficult to meet the computing power requirements for more complex data fusion processing.

[0047] In view of the above technical problems, the present invention provides a method for transmitting data fusion features of heterogeneous devices in Internet of Vehicles to the cloud, the process of which is as follows: Figure 1 As shown, the following steps are included:

[0048] S1. Construct the device identification and data model corresponding to each vehicle-mounted edge device in the Internet of Things in the cloud; the device model corresponding to the vehicle-mounted edge device is used to identify the individual identity of the vehicle-mounted edge device; the data model corresponding to the vehicle-mounted edge device is used to record the working data of the corresponding vehicle-mounted edge device;

[0049] S2. Deploy edge collection and computing devices in the preset data collection area. Each edge collection and computing device can establish a data communication connection with multiple vehicle-mounted edge devices, and build a data transmission channel between the edge collection and computing device and the cloud, so that the working data collected by the vehicle-mounted edge device can be transmitted to the cloud through the edge collection and computing device;

[0050] S3, the vehicle-mounted edge device collects its working data and transmits it to the edge collection and computing device with which it establishes a data communication connection; the working data includes text data, vibration data and audio data of the vehicle-mounted edge device;

[0051] S4. The edge acquisition computing device performs feature extraction and unsupervised fusion processing on the text data, vibration data and audio data in the working data of the vehicle-mounted edge device to obtain the fused feature data of the vehicle-mounted edge device, and uploads it to the cloud through the data transmission channel;

[0052] S5. The cloud side parses and converts the fused feature data of the vehicle-mounted edge device, and obtains the text recovery data, vibration recovery data, and audio recovery data of the vehicle-mounted edge device, which are stored as working parsed data in the data model corresponding to the vehicle-mounted edge device.

[0053] The present invention proposes a method for transmitting the fusion features of data of heterogeneous devices in the Internet of Vehicles to the cloud. In view of the problems that the amount of heterogeneous data of vehicle-mounted edge devices is large and easily causes communication congestion in cloud transmission, the present invention combines a multi-source heterogeneous data fusion algorithm to extract features and perform unsupervised fusion processing on heterogeneous data such as text data, vibration data and audio data, and fusion and reduce their dimensions into fused feature data with low data volume for transmission to the cloud, thereby saving network bandwidth for cloud transmission and reducing its network load. On the other hand, the fused feature data of the vehicle-mounted edge devices is parsed and converted in the cloud and restored to the preset text data, vibration data and audio data format, and the working data of the vehicle-mounted edge devices is stored. In addition, corresponding device identifiers and corresponding data models are respectively set for different vehicle-mounted edge devices, so as to facilitate the distinction of vehicle-mounted edge devices for the stored heterogeneous data such as text, audio, vibration, etc., and provide data support for subsequent equipment operation and maintenance applications. It can be seen that the method of the present invention can reduce the amount of heterogeneous data of vehicle-mounted edge devices when transmitting to the cloud, save network bandwidth for cloud transmission, and reduce its network load. At the same time, it can also well ensure the storage integrity of future data of vehicle-mounted edge devices in the cloud, thereby helping to improve the data interaction efficiency within the Internet of Vehicles system and enhance the overall perception capability of the Internet of Vehicles system of vehicle status information.

[0054] Next, the method for transmitting data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud of the present invention is described in detail.

[0055] Figure 2 The schematic diagram of an example of a method for transmitting data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud in a specific application is shown. This example is also used to demonstrate and explain it in detail below.

[0056] During the specific implementation, in step S1, the device identification mainly plays the role of identifying the identity of the vehicle-mounted edge device, and its specific data form can be diverse, for example, it can be a device ID or a visual device model. In the current application scenario, in order to better meet the high requirements for data visualization, a visual device model can be preferably used as the device identification. In the construction of the device model, it can be based on the object construction specification and adopt an object-oriented concept to realize the rapid construction of the device model of the vehicle-mounted edge device based on the digital abstract model of heterogeneous device objects. As an abstract layer of edge devices, the device model shields the underlying terminal differences, standardizes the device's capability expression and interaction methods, forms a hierarchical object structure, and provides standardized support and guarantee for applications and other services. Use the device model to depict the device structure portrait, and use the data model to describe the device data map, and you can enter the edge device related information in the cloud.

[0057] In the specific implementation of building a device model, in order to improve the efficiency of model reuse, hierarchical modeling can be performed according to device models, device objects, and device components. A device model represents a digital abstract model of a type of equipment and represents a specific device model; a device component represents a certain type of component that is of concern and monitored. The entire device model is subdivided into multiple subsystems, and each individual subsystem is further subdivided into more components. Hierarchical modeling describes the structural portrait of the device model through device components; a device object represents an instantiation of the device model. The device object inherits all the characteristics of the corresponding device model.

[0058] Taking the car as an example, hierarchical modeling is performed in the cloud according to the device model, device object, and device components, and the corresponding configuration information is entered to complete the device registration. The device model is a digital abstract model of a type of car, representing a specific car model; each device model is subdivided into multiple subsystems, such as a car engine, transmission, etc.; each subsystem is composed of multiple components, such as a car transmission spindle, tires, etc. The device object represents an instantiation of the device model, such as a car with a unique ID.

[0059] In the process of building the device model, you need to first build a data map for the device model, which can define the collection of all sampled data sets of the device model. By providing a visual device data model management function and customizing the basic data model, you can implement a structured definition of the data model, and finally bind the data packet model to the device identifier (the above-mentioned device model or device ID, etc.). The data set is divided into the whole machine level and the component level. Each data set requires customized measurement point information. The data type can include vibration signals, audio information, text information, etc. The structural portrait and data map constitute a complete device model. Figure 3 As shown, the device model constructed for the speed change gear box system is shown, taking car A as an example.

[0060] In specific implementation, in step S1, a data storage space for the vehicle-mounted edge device and its components is set in the data model corresponding to the vehicle-mounted edge device, which is used to store the working data of the vehicle-mounted edge device and the working data of its components respectively. The data model can be regarded as a data repository for the device data map, which is used to store the vibration signal, audio information, text information, etc. of the vehicle-mounted edge device.

[0061] In specific implementation, step S2 includes:

[0062] S201, establishing a device access gateway between the cloud and the edge collection and computing device, each device access gateway is used as a data access point for one or more edge collection and computing devices;

[0063] S202. Establish a data access channel in the cloud to serve as a data transmission channel between the cloud and the device access gateway, so that the working data of the vehicle-mounted edge device collected by the edge collection and computing device can be uploaded to the cloud through the device access gateway and the data access channel.

[0064] A new device access gateway is built in the cloud. Each gateway is an access point (endpoint) of the device, representing an abstract device collection access terminal. It can manage several actual vehicle-mounted edge devices and is a unified entrance and exit between the data collection terminal and the platform data access terminal. When configuring the relevant information, it includes: gateway configuration, including the configuration of the MQTT protocol, including the gateway name, region, organization, gateway description, etc.; transmission configuration, including the message identifier of the transmission, the transmission cycle, the transmission delay time, the data expiration time, the data packet size, etc.; data encoding configuration, according to the needs of different devices, configure the corresponding encoding format, such as JSON, binary, text, etc.; data encryption configuration, according to the data requirements of different devices, data can be encrypted before transmission, supporting mainstream symmetric algorithms such as AES and DES, to ensure the network transmission security when the data is sent to the system data exchange layer.

[0065] When creating a new data access channel in the cloud, multiple data channels can be built under each gateway. Each channel can correspond to a data model. For example, if four data access channels are built, four data models can be corresponding to them in the specific design: building a car recording data model, including two data items: time and audio sampling data; building a car log data model, including two data items: time and content; building a gearbox transmission shaft vibration data model, including two data items: time and vibration sampling data; building a car audio and text vibration data fusion feature data model, including two data items: time fusion feature. And so on.

[0066] In specific implementation, in step S3, the working data collected by the vehicle-mounted edge device includes text data, vibration data, and audio data of the vehicle-mounted edge device. Taking a car as an example, the working data collected by a certain model of car may include its whole machine log data log.txt (text data), whole machine recording data audio.wav (audio data), and gearbox vibration data vib.dat (vibration data); for a specific parameter example, for example, the sampling frequency of collecting recording data and vibration data is 10240Hz, and the text content of the log data is "transmission gearbox drive shaft failure, serious abnormal noise".

[0067] When it is implemented specifically, the flowchart of step S4 is as follows Figure 4 As shown, the following steps are included:

[0068] S401, the vehicle-mounted edge device communicates data with the edge collection and computing device, and the edge collection and computing device collects working data of the vehicle-mounted edge device;

[0069] S402, the edge collection and computing device performs preprocessing of word segmentation, sentence segmentation, and stop word removal on the text data in the working data;

[0070] S403, the edge collection and calculation device performs word embedding feature extraction on the preprocessed text data to obtain a word feature vector of the text data;

[0071] S404, the edge acquisition and calculation device performs slicing and segmenting processing on the vibration data and the audio data in the working data in time sequence to obtain a vibration data slice vector and an audio data slice vector;

[0072] S405. The edge acquisition and computing device performs unsupervised dimensionality reduction and fusion processing on the word feature vectors of the text data, the vibration data slice vectors and the audio data slice vectors to obtain fused feature data of the vehicle-mounted edge device.

[0073] In a specific implementation, the edge acquisition computing device can be designed to include a communication unit, a text preprocessing unit, a text feature extraction unit, a time series slicing processing unit and a fusion processing unit. Among them, the communication unit is used to communicate data with the vehicle-mounted edge device and collect the working data of the vehicle-mounted edge device; the text preprocessing unit is used to perform word segmentation, sentence segmentation, and stop word removal preprocessing on the text data in the working data; the text feature extraction unit is used to perform word embedding feature extraction on the preprocessed text data to obtain the word feature vector of the text data; the time series slicing processing unit is used to perform time series analysis on the vibration data and audio data in the working data, and perform slicing and segmentation processing according to the time series to obtain the vibration data slice vector and the audio data slice vector; the fusion processing unit is used to perform unsupervised fusion and dimensionality reduction processing on the word feature vector, vibration data slice vector and audio data slice vector of the text data to obtain the fusion feature data of the vehicle-mounted edge device.

[0074] During specific implementation, the details of step S401 in step S4 are as follows: the vehicle-mounted edge device establishes data connection with the edge collection and computing device that can establish data communication, obtains the data processing resource occupancy rate of the edge collection and computing device, and selects an edge collection and computing device whose data processing resource occupancy rate is less than a preset resource occupancy threshold, and uploads the working data of the vehicle-mounted edge device.

[0075] For example, car A determines that the amount of perceived data is too large, and communicates with the roadside unit through the 4G communication module. It selects an edge collection and computing device whose data processing resource occupancy rate is less than the preset resource occupancy threshold (that is, the communication load and computing load are small) to perform data collection and fusion processing. The data transmission process between them will include initiating communication, identity authentication, establishing connection, data transmission, data confirmation, collaborative computing and other processes.

[0076] In the specific implementation, the details of step S402 in step S4 are: the edge acquisition computing device uses the Bert model that has been pre-trained for text preprocessing to perform word segmentation, sentence segmentation, and stop word removal on the text data in the working data.

[0077] For example, the Bert model has a maximum sequence input limit of 512. The text data is first processed by sentence segmentation, and the text is segmented according to the context semantics and punctuation marks such as ".", "?", "!", ";" are used for auxiliary segmentation to obtain sentence texts with different lengths. Then, for sentence texts of different lengths, Chinese is segmented by characters, and English is segmented by words, and the continuous natural language text is divided into the smallest units (words or subwords) with semantic meaning. Finally, common words that contribute less to understanding the meaning of the text, namely stop words, are removed to reduce noise and redundant information in the text to improve the effect of text processing and analysis. For example, the whole machine log data log.txt of the aforementioned example of "transmission gearbox transmission shaft failure, severe abnormal noise" is obtained after preprocessing as "change", "speed", "device", "tooth", "wheel", "box", "transmission", "motion", "shaft", "fault", "obstacle", "abnormal", "noise", "severe", and "heavy".

[0078] In the specific implementation, the details of step S403 in step S4 are as follows: the edge acquisition computing device uses the pre-trained Bert-base-chinese model to extract word embedding features for Chinese text data, and uses the pre-trained Bert-base-uncased model to extract word embedding features for English text data; the word feature vector representation of the obtained text data is dimensional matrix vector, a single sentence in the text data is represented as n i ×N x dimensional vector, a single character is represented as 1×N x dimensional vector, N x Indicates the feature dimension of feature extraction, n i represents the number of characters in the i-th sentence in the text data, and m represents the number of sentences obtained after the text data is segmented, so that the word feature vector not only contains the character information and semantic information of the global text in the text data, but also contains the position information of each sentence and character.

[0079] Use the open source pre-trained Bert-base-uncased model to process English data and the Bert-base-chinese model to process Chinese data, extract features from text data, and represent text data with word vectors through word vectors, position vectors, and text vectors.

[0080] For example, if the feature dimension N of the feature extraction of the preset Bert model is x =768. After the Bert model, the output representation of a single character is 1×768, and the output representation of a single sentence is n i ×768 dimensions, n iRepresents the number of characters in the i-th sentence, and each sentence is normalized. The vector representation of m sentences is concatenated by the concat method to obtain a single text data. The final vector representation can be expressed as dimensional matrix vector, where m represents the number of sentences obtained after the text data is segmented.

[0081] like Figure 5 As shown, the above-mentioned machine log data log.txt "transmission gearbox transmission shaft failure, serious abnormal noise" is obtained after the Bert model to obtain the word feature vector of 15×768 dimensional vector matrix (including 15 words: "change", "speed", "device", "tooth", "wheel", "box", "transmission", "drive", "shaft", "fault", "obstacle", "abnormal", "noise", "serious", "serious"). As the vector representation of the text data of the whole machine log data log.txt, the word feature vector not only contains the character information and semantic information of the global text in the text data, but also contains the position information of each sentence and character, and has sufficient expression ability.

[0082] In the specific implementation, the details of step S404 in step S4 are as follows: the edge acquisition computing device reads the vibration data and the audio data in the working data respectively and performs normalization processing, and then extracts the feature dimension N according to the preset feature x , the vibration data and audio data are sliced ​​and segmented; the obtained vibration data slice vector is represented as x×N x dimensional vector, the audio data slice vector is represented as y×N x dimensional vectors, x and y represent the number of slice segments of the vibration data and the audio data respectively, so that the vibration data slice vector and the audio data slice vector not only contain the amplitude characteristic information of the vibration data and the audio data respectively, but also contain the position information of each slice segment.

[0083] For example, the vibration data and audio data are read and normalized respectively, and then divided according to the cutting dimension 768 to obtain multiple data blocks, which are used as the vector representation x×768 of the vibration data and the vector representation y×768 of the audio data. After that, the vector representation of the vibration data and the vector representation of the audio data can be connected by concat to obtain a vector matrix of (x+y)×768 dimensions. For example, the audio audio.wav with a duration of 4s is parsed to obtain 4×10240 data points, and the vibration log.dat with a duration of 20s is parsed to obtain 20×10240 data points. After each is normalized, the data is cut according to 768, and finally the insufficient data is filled with data point 1, obtaining two data matrices of 54×768 and 280×768 dimensions respectively, and then concat is used to obtain a 334×768-dimensional audio-vibration joint data matrix.

[0084] In the specific implementation, the details of step S405 in step S4 are as follows: the edge acquisition computing device first performs splicing and fusion processing on the word feature vector of the text data, the vibration data slice vector and the audio data slice vector to obtain the working data feature matrix vector, and then inputs the working data feature matrix vector into the pre-trained Autoencoder self-encoding model for unsupervised dimensionality reduction encoding processing to obtain the fusion feature data of the vehicle-mounted edge device; wherein the working data feature matrix vector is represented as dimensional matrix vector, N x Indicates the feature dimension of feature extraction, n i represents the number of characters in the i-th sentence in the text data, m represents the number of sentences obtained after the text data is segmented, and x and y represent the number of slice segmentations of the vibration data and the audio data, respectively.

[0085] For example, the vector representations of vibration data and audio data are concat- ed with the text representation vector to obtain a 349×768-dimensional working data feature matrix vector, which is used as the input of the Autoencoder model.

[0086] The Autoencoder model is a commonly used dimensionality reduction fusion encoding model. Its architecture is as follows Figure 6 As shown. In the Autoencoder model, the Encoder and Decoder each contain three one-dimensional convolutional layers. The three one-dimensional convolutional layers of the Encoder reduce the input matrix to 256, 64, and 32 dimensions in turn, and obtain a 349×32-dimensional feature matrix for the hidden layer output. The Decoder restores the hidden layer output back to the 349×768-dimensional input vector matrix in turn. Before applying it for vector dimensionality reduction and fusion encoding, the Autoencoder model can be trained by minimizing the reconstruction error MSE between the encoder input and the decoder output to achieve the expected dimensionality reduction and fusion requirements. The calculation formula of the MSE loss function is Among them, y i represents the model output encoding label during training, Indicates the target encoding label of model training, and K indicates the number of training samples. In the specific application implementation, the hyperparameters of the Autoencoder model during training are set as follows: the neural network optimizer is Adam, and the learning rate is 1e -4 , the batch size is 10, the Dropout random inactivation rate is 0.4, and a total of 200 epochs are trained.

[0087] In the specific implementation, step S5 includes: the cloud first decodes the fused feature data of the vehicle-mounted edge device through the Autoencoder self-encoding, and parses it to obtain The working data feature matrix vector of the vehicle-mounted edge device is then decomposed to obtain The word feature vector of the text data of x×N x dimensional vibration data slice vector and y×N x dimensional audio data slice vector, and according to the position information contained in the word feature vector of the text data, the vibration data slice vector and the audio data slice vector, feature reorganization is performed on the word features of the text data and the slice segments of the vibration data and the audio data in position order, and then the respective reorganization results are converted into data formats according to the preset text data format, vibration data format and audio data format, to obtain the text recovery data, vibration recovery data and audio recovery data of the vehicle-mounted edge device, and finally the text recovery data, vibration recovery data and audio recovery data of the vehicle-mounted edge device are used as the working parsing data of the corresponding vehicle-mounted edge device, and stored in the data model corresponding to the corresponding vehicle-mounted edge device.

[0088] The constructed data transmission channel is used to transmit the data features fused at the edge to the cloud, and provide key authentication parameters; the edge uses the industrial gateway to establish communication with the cloud, and efficiently transmits the fused data features to the cloud; the original data or pre-processed heterogeneous data can be sent when communication resources are idle, and collected in the cloud data center for backup.

[0089] Based on the above data architecture layout between the cloud and the vehicle-mounted edge devices and the method of transmitting to the cloud, in order to achieve high-compatibility data access between the vehicle-mounted edge devices and the cloud, the high-compatibility access task of heterogeneous data of the vehicle-mounted edge devices is decoupled into data access and transmission, data analysis and processing, and data storage and management. Specifically, the overall data transmission process to the cloud can be summarized as follows:

[0090] Data access and transmission: Establish an industrial gateway based on the constructed device data model. The industrial data gateway is the network interface for data access to the IoT cloud platform, responsible for providing a network transmission channel for the collected data to be sent to the IoT cloud platform. Using Mosquitto open source message agent software and MQTT data transmission protocol, the collected data is received in the standard JSON format and sent to the Kafka message queue for asynchronous processing and current limiting and peak shaving.

[0091] Data parsing and processing: After the cloud receives the data transmitted by the device, it needs to be parsed and processed. According to the unified object model, the device data is parsed into standard formats of vibration data, audio data, and text data, and data cleaning, verification, conversion and other processing are performed to facilitate subsequent data storage and analysis. The original data in the internal message queue is decrypted, and the decrypted data is parsed into the specified data structure according to the data access model.

[0092] Data storage and management: The parsed heterogeneous device data can be stored in the MongoDB database in the cloud for subsequent query and analysis. The data storage and data management framework designed and developed based on the loosely coupled technical design concept can realize structured data and unstructured data storage, and can be integrated with third-party data platforms through data adaptation, such as the Hadoop big data platform.

[0093] In summary, the method of the present invention has the following technical advantages:

[0094] 1. The present invention proposes a method for transmitting the data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud. In view of the large amount of heterogeneous data of vehicle-mounted edge devices, which easily causes communication congestion in cloud transmission, the method combines a multi-source heterogeneous data fusion algorithm to extract features and perform unsupervised fusion processing on heterogeneous data such as text data, vibration data and audio data, and then reduces the dimensionality of the fused feature data with low data volume for transmission to the cloud, thereby saving network bandwidth for cloud transmission and reducing its network load.

[0095] 2. In the method for transmitting the fusion feature data of heterogeneous devices in the Internet of Vehicles to the cloud of the present invention, the fusion feature data of the vehicle-mounted edge device is analyzed and converted in the cloud and restored to the preset text data, vibration data and audio data format, and the working data of the vehicle-mounted edge device is stored. In addition, corresponding device identifiers and corresponding data models are set for different vehicle-mounted edge devices, which facilitates the differentiation of vehicle-mounted edge devices for the stored heterogeneous data such as text, audio, vibration, etc., and provides data support for subsequent equipment operation and maintenance and other applications.

[0096] 3. The method of the present invention can reduce the amount of heterogeneous data of vehicle-mounted edge devices when transmitting them to the cloud, save network bandwidth for cloud transmission, and reduce its network load. At the same time, it can also well ensure the storage integrity of future data of vehicle-mounted edge devices in the cloud, thereby helping to improve the data interaction efficiency within the Internet of Vehicles system and enhance the overall perception capability of the Internet of Vehicles system of vehicle status information, and has great value for technology promotion and application.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. A method for transmitting data fusion features of heterogeneous devices in Internet of Vehicles to the cloud, characterized in that: The steps include: S1. Construct the device identification and data model corresponding to each vehicle-mounted edge device in the Internet of Things in the cloud; the device model corresponding to the vehicle-mounted edge device is used to identify the individual identity of the vehicle-mounted edge device; the data model corresponding to the vehicle-mounted edge device is used to record the working data of the corresponding vehicle-mounted edge device; S2. Deploy edge collection and computing devices in the preset data collection area. Each edge collection and computing device can establish a data communication connection with multiple vehicle-mounted edge devices, and build a data transmission channel between the edge collection and computing device and the cloud, so that the working data collected by the vehicle-mounted edge device can be transmitted to the cloud through the edge collection and computing device; S3, the vehicle-mounted edge device collects its working data and transmits it to the edge collection and computing device with which it establishes a data communication connection; the working data includes text data, vibration data and audio data of the vehicle-mounted edge device; S4: The edge acquisition computing device performs feature extraction and unsupervised fusion processing on the text data, vibration data and audio data in the working data of the vehicle-mounted edge device to obtain the fused feature data of the vehicle-mounted edge device, and uploads it to the cloud through the data transmission channel; step S4: S4 includes: S401, the vehicle-mounted edge device communicates data with the edge collection and computing device, and the edge collection and computing device collects working data of the vehicle-mounted edge device; S402, the edge collection and computing device performs preprocessing of word segmentation, sentence segmentation, and stop word removal on the text data in the working data; S403, the edge collection and calculation device performs word embedding feature extraction on the preprocessed text data to obtain a word feature vector of the text data; S404, the edge acquisition and calculation device performs slicing and segmenting processing on the vibration data and the audio data in the working data in time sequence to obtain a vibration data slice vector and an audio data slice vector; S405, the edge acquisition computing device performs unsupervised dimensionality reduction fusion processing on the word feature vectors of the text data, the vibration data slice vectors and the audio data slice vectors to obtain the fused feature data of the vehicle-mounted edge device; in step S405, the edge acquisition computing device first performs splicing and fusion processing on the word feature vectors of the text data, the vibration data slice vectors and the audio data slice vectors to obtain the working data feature matrix vector, and then inputs the working data feature matrix vector into the pre-trained Autoencoder self-encoding model for unsupervised dimensionality reduction encoding processing to obtain the fused feature data of the vehicle-mounted edge device; wherein the working data feature matrix vector is represented as dimensional matrix vector, N x Indicates the feature dimension of feature extraction, n i represents the number of characters in the i-th sentence in the text data, m represents the number of sentences obtained after the text data is segmented, and x and y represent the number of slice segmentation segments of the vibration data and the audio data, respectively; S5. The cloud side parses and converts the fused feature data of the vehicle-mounted edge device, and obtains the text recovery data, vibration recovery data, and audio recovery data of the vehicle-mounted edge device, which are stored as working parsed data in the data model corresponding to the vehicle-mounted edge device.

2. According to claim 1, the method for transmitting data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud is characterized by: In the step S1, a data storage space for the vehicle-mounted edge device and its components is set in the data model corresponding to the vehicle-mounted edge device, which is used to store the working data of the vehicle-mounted edge device and the working data of its components respectively.

3. According to claim 1, the method for transmitting data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud is characterized by: The step S2 comprises: S201, establishing a device access gateway between the cloud and the edge collection and computing device, each device access gateway is used as a data access point for one or more edge collection and computing devices; S202. Establish a data access channel in the cloud to serve as a data transmission channel between the cloud and the device access gateway, so that the working data of the vehicle-mounted edge device collected by the edge collection and computing device can be uploaded to the cloud through the device access gateway and the data access channel.

4. According to claim 1, the method for transmitting data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud is characterized by: In step S401, the vehicle-mounted edge device establishes data connection with an edge acquisition and computing device that can establish data communication, obtains the data processing resource occupancy rate of the edge acquisition and computing device, and selects an edge acquisition and computing device whose data processing resource occupancy rate is less than a preset resource occupancy threshold to upload the working data of the vehicle-mounted edge device.

5. According to claim 1, the method for transmitting data fusion features of heterogeneous devices in the Internet of Vehicles to the cloud is characterized by: In the step S402, the edge acquisition computing device uses the Bert model that has been pre-trained for text preprocessing to perform word segmentation, sentence segmentation, and stop word removal on the text data in the working data.

6. The method for transmitting data fusion features of heterogeneous devices in Internet of Vehicles to the cloud according to claim 1 is characterized in that: In step S403, the edge acquisition computing device uses a pre-trained Bert-base-chinese model to perform word embedding feature extraction on Chinese text data, and uses a pre-trained Bert-base-uncased model to perform word embedding feature extraction on English text data; The word feature vector representation of the obtained text data is dimensional matrix vector, a single sentence in the text data is represented as n i ×N x dimensional vector, a single character is represented as 1×N x dimensional vector, N x Indicates the feature dimension of feature extraction, n i represents the number of characters in the i-th sentence in the text data, and m represents the number of sentences obtained after the text data is segmented, so that the word feature vector not only contains the character information and semantic information of the global text in the text data, but also contains the position information of each sentence and character.

7. The method for transmitting data fusion features of heterogeneous devices in Internet of Vehicles to the cloud according to claim 1 is characterized in that: In step S404, the edge acquisition computing device reads the vibration data and the audio data in the working data respectively and performs normalization processing, and then extracts the feature dimension N according to the preset feature. x , the vibration data and audio data are sliced ​​and segmented; the obtained vibration data slice vector is represented as x×N x dimensional vector, the audio data slice vector is represented as y×N x dimensional vectors, x and y represent the number of slice segments of the vibration data and the audio data respectively, so that the vibration data slice vector and the audio data slice vector not only contain the amplitude characteristic information of the vibration data and the audio data respectively, but also contain the position information of each slice segment.

8. The method for transmitting data fusion features of heterogeneous devices in Internet of Vehicles to the cloud according to claim 1 is characterized in that: In step S5, the cloud first decodes the fused feature data of the vehicle-mounted edge device through the Autoencoder self-encoding process to obtain The working data feature matrix vector of the vehicle-mounted edge device is then decomposed to obtain The word feature vector of the text data of x×N x The y×N-dimensional vibration data slice vector and x dimensional audio data slice vector, and according to the position information contained in the word feature vector of the text data, the vibration data slice vector and the audio data slice vector, feature reorganization is performed on the word features of the text data and the slice segments of the vibration data and the audio data in position order, and then the respective reorganization results are converted into data formats according to the preset text data format, vibration data format and audio data format, to obtain the text recovery data, vibration recovery data and audio recovery data of the vehicle-mounted edge device, and finally the text recovery data, vibration recovery data and audio recovery data of the vehicle-mounted edge device are used as the working parsing data of the corresponding vehicle-mounted edge device, and stored in the data model corresponding to the corresponding vehicle-mounted edge device.

Images