Data analysis method and system and medium

Abstract

The invention discloses a data analysis method and system and a medium, and the method comprises the steps: obtaining a communication mode of a sensor, and configuring the communication mode based on a preset configuration file; determining a data analysis mode according to the communication mode; analyzing the sensor data according to the data analysis mode to obtain analyzed sensor data; mapping the analyzed sensor data into a sensor abstract signal; and transmitting the sensor abstract signal. According to the method, abstraction processing can be carried out on different types of sensor data, so that an upper application can read the data of different sensors in a simple and uniform mode.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data analysis method, system and medium. Background Art

[0002] With the increasing intelligence and automation of automobiles, more and more types of sensors are being used in automobiles. Vehicle cameras, millimeter-wave radars, ultrasonic radars, lidars, etc. play an important role in the fields of ADAS and autonomous driving.

[0003] Different types of sensors have different working principles, detection indicators and data transmission methods. How to unify the interface and achieve cross-platform portability is the goal of exploration within the industry. Summary of the invention

[0004] In response to the above technical problems, the present invention provides a data analysis method, system and medium, which can abstract and unify different types of sensor data, so that upper-level applications do not need to worry about the usage differences between sensors and can read data from different sensors in a simple and unified manner.

[0005] A first aspect of the present invention provides a data analysis method, comprising: Acquire a communication mode of the sensor, and configure the communication mode based on a preset configuration file; Determining a data parsing method according to the communication method; Parsing the sensor data according to the data parsing method to obtain parsed sensor data; Mapping the parsed sensor data into a sensor abstract signal; And, sending the sensor abstract signal.

[0006] In a possible implementation manner, determining the data parsing manner according to the communication manner includes: If the communication mode is CAN communication, the parsing mode of the CAN signal is configured according to the preset ARXML file; If the communication mode is not CAN communication, the serialization / deserialization mode is configured according to a preset custom data format.

[0007] In a possible implementation manner, determining the data parsing manner according to the communication manner includes: If the data analysis method is a CAN signal analysis method, the sensor signal is converted into a sensor abstract signal; If the data parsing method is a serialization / deserialization method, the sensor data is serialized or deserialized.

[0008] In a possible implementation, configuring the parsing method of the CAN signal according to the preset ARXML file includes: Import CAN ARXML file; Generate DBC files corresponding to each CAN channel; Create CAN data parsing code based on the DBC file.

[0009] In a possible implementation, the creating a CAN data parsing code according to the DBC file includes: Input CAN payload data; Parse the CAN payload data to obtain the raw value; The raw value is parsed to obtain the physical value.

[0010] In a possible implementation, configuring the serialization / deserialization method according to the preset custom data format includes: Define the data structure through an Excel table with custom data format; Parsing the data structure into a corresponding ARXML file; Generate serialization and deserialization codes corresponding to the ARXML file.

[0011] In a possible implementation manner, configuring the communication mode based on a preset configuration file includes: Reading a sensor name from the preset configuration file; Create a corresponding working thread according to the sensor name.

[0012] In a possible implementation, the creating a corresponding working thread according to the sensor name includes: Confirming the data parsing method according to the data type in the preset configuration file; The data transmission mode is confirmed according to the trans type in the preset configuration file.

[0013] In a possible implementation manner, mapping the parsed sensor data into a sensor abstract signal includes: Defining a mapping relationship of sensor data according to a preset sensor signal mapping table; Generate the corresponding abstract transformation code.

[0014] In a possible implementation, the generating of the corresponding abstract conversion code includes: Convert sensor data into abstract data; Alternatively, convert abstract data into concrete sensor data.

[0015] A second aspect of the present invention provides a data analysis system, comprising: A communication configuration module, at least used to obtain a communication mode of the sensor and configure the communication mode based on a preset configuration file; A data analysis module, at least used to determine a data analysis method according to the communication method, and analyze the sensor data according to the data analysis method to obtain analyzed sensor data; A data abstraction module, at least used to map the parsed sensor data into a sensor abstract signal; The signal sending module is at least used to send the sensor abstract signal.

[0016] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a computer, the method according to the first aspect of the embodiment of the present invention is executed.

[0017] The present invention achieves flexible support for multiple communication protocols (such as CAN, IPCF, Ethernet, etc.) by acquiring the communication mode of the sensor and automatically configuring it based on the configuration file. It can select the appropriate data parsing mode according to different protocols, uniformly parse various sensor data and map them into abstract signals, simplifying the development of upper-level applications. Compared with the prior art, the present invention significantly improves the efficiency and reliability of data parsing by tooling to generate parsing codes and configuration files, enhances the portability and scalability of the system, adapts to the integration requirements of multiple sensor data, and effectively supports data interaction in complex scenarios such as ADAS and autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flowchart of an embodiment of a data analysis method of the present invention.

[0019] Figure 2 The present invention is a flowchart of an embodiment of a data analysis method.

[0020] Figure 3 The present invention is a flowchart of another embodiment of a data analysis method.

[0021] Figure 4 The present invention is a flowchart of another embodiment of a data analysis method.

[0022] Figure 5 The present invention is a flowchart of another embodiment of a data analysis method.

[0023] Figure 6 The present invention is a flowchart of another embodiment of a data analysis method.

[0024] Figure 7 It is a structural schematic diagram of an embodiment of a data analysis system of the present invention.

[0025] Figure 8 This is an application scenario diagram of an embodiment of a data analysis system of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0027] It should be understood that the terms "first", "second", and "third" etc. in the claims, specifications, and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections. It should also be understood that the terms used in this disclosure specification are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0028] Sensors play an important role in the field of smart cars and autonomous driving, including various types such as on-board cameras, millimeter-wave radars, ultrasonic radars and lidars. On-board cameras are used to provide visual information around the vehicle, such as reversing images and panoramic views; millimeter-wave radars can work all day and have strong penetration and high resolution, and are suitable for detecting small target objects; ultrasonic radars detect obstacles through ultrasonic waves and are often used for assisted driving functions; lidars obtain environmental data through high-precision laser scanning, which is used to achieve accurate target detection and environmental perception. Different types of sensors differ in data transmission and analysis methods.

[0029] Different types of sensors have differences in data transmission and analysis methods, which are mainly reflected in the differences in data format, protocol type and analysis method. For example, vehicle-mounted cameras usually transmit high-bandwidth video data through Ethernet (ETH), which requires specific image analysis algorithms; millimeter-wave radars usually use CAN bus transmission, with a small amount of data but requiring real-time signal processing; ultrasonic radars also mostly use CAN bus transmission to transmit short-distance obstacle data, and the analysis process is relatively simple; laser radars usually transmit high-resolution point cloud data through Ethernet or SPI interfaces, and the analysis method is complex, requiring precise spatial positioning and shape recognition. This difference leads to the complexity of data analysis and transmission, and unified interfaces and analysis methods are technically challenging.

[0030] Based on the above problems, refer to Figure 1 , an embodiment of the present invention provides a data analysis method, comprising the following steps.

[0031] S1, obtaining a communication mode of a sensor, and configuring the communication mode based on a preset configuration file.

[0032] Specifically, the communication mode of the currently connected sensor can be automatically detected by reading the configuration file or sensor properties, such as CAN bus, Ethernet (ETH), IPCF (inter-chip communication), SPI, etc. Depending on the communication mode, the corresponding communication parameters are loaded from the preset configuration file (such as JSON or XML format).

[0033] In some implementations, the communication parameters include the baud rate and node ID of the CAN bus; the IP address, port number, and transmission rate of the Ethernet; the clock frequency, data format, and working mode of the SPI, etc. For the IPCF communication mode, the configuration file defines the parameters of the shared memory address or message queue.

[0034] After loading is completed, initial configuration is performed according to the parameters in the configuration file so that the communication methods of various sensors adapt to system requirements and ensure smooth data transmission. In addition, step S1 may also include a verification step to confirm that the loaded configuration parameters match the actual hardware connection.

[0035] S2, determining a data parsing method according to the communication method.

[0036] Specifically, choose the appropriate data parsing method according to different communication modes. For CAN bus communication mode, you can use the parsing code generated by the ARXML file to parse the original CAN message data into physical signal values; for Ethernet communication, you can use the protobuf serialization protocol to parse the data stream and convert the received byte stream into readable sensor data; if the communication mode is SPI, you may use a specific frame structure parsing method to split each frame of data into sensor information; for IPCF communication, you can use the shared memory parsing tool to directly read the sensor data from the shared memory and then perform data conversion.

[0037] At the same time, the corresponding parsing library or plug-in can be loaded according to the protocol type specified in the configuration file to support customized parsing requirements, thereby ensuring that data under different communication modes can be decoded smoothly.

[0038] S3, analyzing the sensor data according to the data analysis method to obtain analyzed sensor data.

[0039] Specifically, the sensor data is processed according to different data analysis methods to obtain structured analysis data.

[0040] In some implementations, for data transmitted using the CAN bus, a parsing function generated by an ARXML file may be called to convert the original CAN message into corresponding physical quantities, such as speed, distance, etc.

[0041] In some implementations, for data transmitted via Ethernet, a serialization protocol such as protobuf or JSON may be used to deserialize the byte stream and restore the binary data to a sensor signal object.

[0042] In some implementations, for sensor data communicated via SPI, the transmitted frame data may be parsed into various field values ​​according to the frame format definition.

[0043] In some implementations, for data in the IPCF shared memory, the memory block may be directly read, and the data may be segmented and decoded into valid information according to a predetermined structure.

[0044] In addition, in some implementations, if special data processing logic is specified in the configuration file, the corresponding parsing plug-in or custom function can be loaded to perform specific conversion or calibration on the data to ensure that data from different sources can be decoded into a unified format.

[0045] S4, mapping the parsed sensor data into sensor abstract signals.

[0046] Specifically, the parsed sensor data is mapped into a unified abstract signal for use by upper-layer applications.

[0047] In some embodiments, each parsed data field can be mapped to a standardized abstract signal name, such as speed, position, temperature, etc., through a preset mapping table (such as Excel or JSON format), ensuring that similar data from different sensors are represented uniformly.

[0048] In some implementations, the data structure may be converted into a ROS message format according to the sensor type and data attributes, and the parsed sensor information may be encapsulated into a standard ROS message (such as lines msg for lane information and objects msg for obstacle information) for easy transmission in the ROS system.

[0049] In some implementations, a custom abstract signal interface can be set up based on the needs of intra-domain or inter-domain applications, so that each data field can be directly subscribed to and used by upper-level applications, avoiding data differences between different sensors from affecting upper-level logic and achieving data abstraction and standardized processing.

[0050] S5, sending the sensor abstract signal.

[0051] Specifically, the abstracted sensor signals are sent to different application modules or system interfaces.

[0052] In some implementations, for a ROS-based system, the abstract signal may be directly published in the form of a ROS message, so that other nodes may subscribe to and use the sensor data in real time.

[0053] In some implementations, if it is a cross-domain system or a scenario that requires remote transmission, abstract signals can be published via Ethernet or DDS protocol to achieve efficient data distribution and remote access.

[0054] In some implementations, in an IPC communication scenario, abstract signals may be transmitted via shared memory or message queues to reduce latency and improve real-time performance of data.

[0055] In some implementations, the abstract signal can also be serialized into a specific format (such as JSON or protobuf) according to the settings of the configuration file, and transmitted to the upper-level application or other devices through an API interface or file export to ensure the compatibility and flexibility of data transmission and meet the needs of different application scenarios.

[0056] Further, as an embodiment of the present invention, in step S2, determining the data parsing method according to the communication method includes: if the communication method is CAN communication, configuring the parsing method of the CAN signal according to a preset ARXML file; if the communication method is not CAN communication, configuring the serialization / deserialization method according to a preset custom data format.

[0057] In some embodiments, when the communication mode is CAN communication, an ARXML file is loaded, which contains configuration information such as the definition of the CAN signal, message ID, signal name, data type, start bit, length and physical unit; according to the signal attributes defined in the ARXML file, a parsing code is generated or an existing CAN parsing library is called to automatically decode the received CAN message into a physical signal value, such as converting the raw data of the sensor into actual physical quantities such as speed, rotation speed, temperature, etc.

[0058] In addition, in some implementations, the parsed data may be subjected to integrity checking through check bits and error detection fields to ensure the reliability of data transmission.

[0059] In some implementations, multiple CAN protocol versions, such as CAN FD, may also be supported, and different parsing requirements may be adapted according to the protocol version and specific fields in the ARXML file to achieve compatible parsing of CAN data from multiple sensors.

[0060] In some embodiments, when the communication method is not CAN communication, the serialization / deserialization method is configured according to a preset custom data format, and the data definition is loaded from a configuration file (such as JSON, XML or protobuf format) to determine the type, structure and length of the data field.

[0061] In some implementations, the communication method is Ethernet, and a serialization protocol such as protobuf or JSON is used to parse the received byte stream into a structured data object.

[0062] In some implementations, the communication method is SPI, and the structure and field attributes of the data frame are parsed through a frame definition table to restore each frame of data to a specific sensor signal.

[0063] In some implementations, the communication method is IPCF or DDS, and the data is parsed into a standardized signal form according to the data structure definition in the shared memory or message queue.

[0064] In addition, in some implementations, custom serialization methods can also be supported, such as compression protocols optimized for embedded systems, to achieve efficient transmission and decoding according to user needs. These methods ensure the flexibility and consistency of data analysis under multiple communication protocols.

[0065] Further, as an implementation mode of the present invention, step S2, determines the data parsing method according to the communication method to obtain the parsed sensor data, including: if the data parsing method is a CAN signal parsing method, the sensor signal is converted into a sensor abstract signal; if the data parsing method is a serialization / deserialization method, the sensor data is serialized or deserialized.

[0066] Specifically, in some embodiments, when the data parsing method is the CAN signal parsing method, the original CAN message data is received, and the signal fields, such as the start bit, data length, scaling factor, and offset, are extracted according to the signal mapping rules defined in the preset ARXML file; then, the extracted signal values ​​are converted into corresponding physical values, such as distance, speed, temperature, etc., through the parsing code; next, according to the definition of the abstract signal, these physical signals are mapped to a unified abstract signal interface, such as mapping the vehicle speed signal to "vehicle_speed", and the obstacle distance signal to "obstacle_distance", etc.; finally, these abstract signals can be further encapsulated into a standardized data format (such as ROS messages, JSON objects) for subscription and use by upper-layer applications or other modules. This method realizes the conversion from the original CAN signal to the cross-platform, cross-protocol abstract signal, ensuring the versatility and ease of use of the data.

[0067] In some embodiments, when the data parsing method is serialization / deserialization, the corresponding serialization / deserialization rules are loaded from the configuration file according to the communication protocol (such as Ethernet, DDS or IPCF) and the preset data format (such as protobuf, JSON or custom binary format); if the received data is in a serialized format (such as a byte stream), the deserialization function is called to restore the byte stream to a structured data object, such as multi-field sensor information; conversely, if data needs to be sent, the serialization function is called to encode the structured data object into a byte stream to adapt to the transmission requirements of the communication protocol.

[0068] In addition, in some implementations, custom serialization methods can be supported for different scenarios, such as compression algorithms optimized for embedded devices to reduce the bandwidth occupied by data transmission. The entire process generates serialization codes through tooling or automation to ensure efficient, flexible and reliable data parsing in a multi-protocol environment.

[0069] For example, refer to Figure 2, a data parsing method of the present invention, an embodiment of the process for abstracting sensor data includes: reading the communication mode configuration from a JSON configuration file, or loading the data definition from a custom Excel data format table. If the communication mode is CAN, the parsing method of the CAN signal is configured according to the ARXML file, and the CAN signal is parsed into physical quantity data; otherwise, the serialization / deserialization method is used to process the data of non-CAN communication. The parsed sensor signal is converted into an abstract signal according to a mapping table, and finally these abstract signals are sent out for use by other modules or systems. This process ensures that data of different communication protocols can be correctly parsed and uniformly converted into abstract signals.

[0070] Further, as an implementation mode of the present invention, in step S1, the communication mode is configured based on a preset configuration file, including: reading a sensor name from the preset configuration file; and creating a corresponding working thread according to the sensor name.

[0071] Further, as an implementation mode of the present invention, the creating a corresponding working thread according to the sensor name includes: confirming a data parsing method according to a data type in the preset configuration file; and confirming a data transmission method according to a trans type in the preset configuration file.

[0072] Specifically, read the name, communication protocol, baud rate or IP address of each sensor from the configuration file (such as JSON or XML); create an independent working thread for each sensor according to the read sensor name, and these threads will be responsible for receiving, parsing and processing the data of each sensor; for CAN communication, the thread can be set to listen to a specific CAN channel; for Ethernet communication, the thread establishes a connection according to the configured IP address and port number; for SPI or IPC communication, the thread is configured to process a specific device file or shared memory area. Each working thread initializes a specific communication mode and resource management according to the parameters in the configuration file, thereby ensuring the parallel execution of communication configuration and data processing of each sensor.

[0073] In some embodiments, the data type of each sensor is read from a preset configuration file, and a corresponding parsing method is selected according to the data type. For example, if the data type is "can", the CAN protocol parsing function is selected; if it is "proto", protobuf or a custom deserialization function is selected.

[0074] In some embodiments, the data transmission mode is confirmed according to the trans type in the configuration file. For example, if the trans type is "eth", the Ethernet communication is initialized and the IP address and port are configured; if it is "ipcf", the inter-process communication (IPC) channel is set; if it is "spi", the SPI interface is initialized and the corresponding hardware parameters are configured.

[0075] Through these settings, each working thread can independently handle the data reception and parsing of its own sensor, ensuring the flexibility and scalability of the system.

[0076] Exemplarily, when the configuration file is a JSON file, refer to Figure 3 , the JSON configuration file example on the left defines the communication and data type configuration of multiple sensors. Each sensor is identified by its name (such as "chcanfd" and "uss"), and the configuration items include the communication data type (data type) and the transmission mode (trans type). For example, the data type of the "chcanfd" sensor is "can", which means that CAN communication is used, and the trans type is "eth", which means that the transmission mode is Ethernet, and it contains the host IP address (host-ip), client IP address (guest-ip) and port number (port) for configuring network communication parameters; the data type of the "uss" sensor is "proto", and the trans type is "ipcf", which means that IPCF transmission is used, and the communication channel (channel) path is specified. This configuration method can flexibly define the communication method and parameters of the sensor, so that the system can automatically load and initialize the communication configuration of each sensor.

[0077] Continue to refer to Figure 3 , the right side shows the process of configuring the communication method based on the above JSON configuration file. First, read the sensor name from the configuration file and create an independent working thread for each sensor. Next, determine the parsing method according to the data type, such as selecting the corresponding protocol parsing library or function; determine the data transmission method according to the trans type, and configure the corresponding communication interface (such as Ethernet, IPCF, etc.); finally, set the transmission parameters according to other information in the configuration file (such as IP address, port, communication channel, etc.), ensure that the communication method and data transmission method of each sensor meet the preset requirements, and complete the initialization. This process realizes automated sensor communication configuration and supports multiple communication methods.

[0078] Further, as an implementation mode of the present invention, in step S2, the parsing method of the CAN signal is configured according to the preset ARXML file, including: importing the CAN ARXML file; generating a DBC file corresponding to each CAN; and creating a CAN data parsing code according to the DBC file.

[0079] The step of creating a CAN data parsing code according to the DBC file includes: inputting CAN payload data; parsing the CAN payload data to obtain a raw value; and parsing the raw value to obtain a physical value.

[0080] Specifically, the parsing method of configuring CAN signals according to the preset ARXML file includes: importing the CAN ARXML file, which contains the definition and parameter configuration of all signals in the CAN network, such as the start bit, length, signal name and physical unit of the signal; then, converting the definition of each CAN signal from the ARXML file to the corresponding DBC file, which is the standard description file of the CAN signal, to facilitate subsequent parsing; based on the generated DBC file, using an automated tool or code generator to create a CAN data parsing code. The parsing code will convert the received raw CAN data into specific physical values, such as parsing the hexadecimal data sent by the sensor into actual temperature or speed values.

[0081] In addition, in some implementations, the corresponding checksum processing logic can be generated according to the check bit and signal mapping relationship defined in the DBC file to ensure the accuracy and integrity of the data. In this way, the system realizes the automatic generation from ARXML files to parsing codes, effectively reducing the workload of manual configuration and coding.

[0082] In some embodiments, creating a CAN data parsing code based on a DBC file includes: receiving and inputting CAN payload data (i.e., the payload data of a CAN message), which is typically a piece of raw binary or hexadecimal encoded information; then, according to the signal mapping and position information defined in the DBC file, parsing the raw value of each signal from the CAN payload, wherein the raw values ​​are unscaled integers or floating point numbers; then, according to the scaling factor and offset in the DBC file, converting the raw value into a physical value, for example, converting the raw temperature value into actual degrees Celsius through calculation, or converting the raw speed value into a physical quantity of kilometers per hour.

[0083] In addition, in some implementations, corresponding checksum and exception handling logic can be generated based on the units and range limits in the DBC file to ensure the accuracy and validity of the data. This approach achieves fully automatic parsing of CAN data from raw binary to human-readable physical quantities.

[0084] For example, refer to Figure 4 The process of generating CAN data parsing code based on CAN ARXML file includes: importing CAN ARXML file, which contains signal definition and configuration of CAN network. Then, generate corresponding DBC file (standard format for describing CAN signal) for each CAN signal, and convert detailed information of the signal into DBC file format. Then, according to the generated DBC file, use automation tool or code generator to generate code for parsing CAN data, so that after receiving CAN message, the data can be automatically parsed into physical quantity. Finally, these parsing codes realize the conversion from raw CAN data to readable physical value.

[0085] Among them, a CAN data parsing code is created according to the DBC file, including obtaining a raw value by parsing the payload after receiving the CAN payload data, and then converting the raw value into a physical value according to a scaling factor and an offset to output actual physical data.

[0086] In some implementations, the reverse operation is also supported, that is, after receiving the physical value, it is first converted into a raw value by reverse parsing, and then the corresponding payload data is generated, and finally the CAN payload is output.

[0087] This bidirectional process can flexibly handle data encoding and decoding, realize the conversion of physical quantities of CAN messages to raw data and vice versa, and ensure the accuracy and consistency of data transmission.

[0088] Furthermore, as an implementation mode of the invention, in step S2, the serialization / deserialization method is configured according to a preset custom data format, including: defining a data structure through an Excel table in a custom data format; parsing the data structure into a corresponding ARXML file; and generating serialization and deserialization codes corresponding to the ARXML file.

[0089] Specifically, in some embodiments, defining a data structure through an Excel table with a custom data format includes: creating a data structure definition table in an Excel table, which contains information such as field name, data type, length, precision, unit and description, so as to clearly define the attributes of each data field.

[0090] In some implementations, separate table pages may be used to define complex data structures, such as nested structures or multi-dimensional arrays.

[0091] In some implementations, the Excel table may contain an enumerated type definition that lists the possible range of values ​​for a particular field.

[0092] In some implementations, the definition of a checksum calculation method may also be added to perform integrity checking during data transmission.

[0093] By parsing the Excel table, a data structure definition that meets the requirements can be automatically generated, providing basic data for subsequent ARXML file generation and serialization / deserialization code generation. This method supports customized and diversified data structure configuration through the flexibility of Excel, which is convenient for rapid adjustment and expansion.

[0094] In some implementations, parsing the data structure into a corresponding ARXML file includes: reading the data structure defined in the Excel table, parsing field name, data type, length, unit and other information item by item, and mapping them into elements of the ARXML file according to the AUTOSAR standard format.

[0095] In some implementations, nested structures and array types may be automatically converted to corresponding composite data types or container elements in an ARXML file.

[0096] In some implementations, for a field that defines an enumeration type, it may be converted into an enumeration value definition in ARXML.

[0097] In some implementations, the constraints in the ARXML file may also be generated according to the validation rules or limited ranges in Excel.

[0098] In some complex scenarios, it is also possible to distribute the data structure in different ARXML files in a modular manner according to the hierarchical structure of the Excel table to ensure the readability and logical layering of the files.

[0099] These implementations achieve automatic conversion of custom data structures into standard ARXML format, reduce manual configuration errors, and improve development efficiency.

[0100] In some implementations, generating serialization and deserialization codes corresponding to the ARXML file includes: reading a data structure and a signal mapping defined in the ARXML file, and automatically generating serialization and deserialization codes matching the data structure and the signal mapping.

[0101] In some implementations, for simple data types, basic encoding and decoding functions are generated to convert structured data to binary format or restore from binary to structured data.

[0102] In some implementations, for nested structures, arrays, or composite types, recursive or iterative serialization code is generated to ensure complete encoding of multi-level data structures.

[0103] In some implementations, for data fields containing enumerations or constraints, the generated code performs a legality check to ensure that the decoded data conforms to the specified value range.

[0104] In addition, in some implementations, specific packing and unpacking codes may be generated according to the requirements of specific communication protocols (such as TCP, UDP, SPI, etc.) to ensure that the data is in the correct format during transmission.

[0105] The serialization and deserialization code automatically generated by these implementations improves data processing efficiency and accuracy and reduces the risk of errors in manual coding.

[0106] For example, refer to Figure 5 , the example of the custom data format Excel table shows the data description format between the MPU (microprocessor unit) and the MCU (microcontroller unit). The table includes information such as signal name (MPU Signal Name), signal element (Signal Element), data size (size), data type (Element DataType), service parameter (CPService Parameter), etc., which is used to define and describe the signals transmitted between the MPU and the MCU. The MPU and MCU sides respectively define the field name and data type of the signal, such as PSL_FunctionStatus, to unify the data structure and format in communication and ensure the consistency of the parsing and transmission of each signal at both ends. This definition of Excel table formatting facilitates the subsequent automatic generation of ARXML files and serialization / deserialization code.

[0107] Continue to refer to Figure 5 The definition and processing flow of a custom data format includes: defining the data structure through an Excel table and generating an ARXML file containing these data structures; then, using tools to generate corresponding serialization and deserialization codes based on the ARXML file to achieve data encoding and decoding. During the data transmission process, the original data is serialized and converted into a format suitable for transmission, and the receiving end restores the data to the original format through deserialization. The entire process realizes automated processing from custom data structure definition to code generation and data transmission, ensuring the compatibility and consistency of data between different systems.

[0108] Further, as an implementation mode of the present invention, step S4, mapping the parsed sensor data into a sensor abstract signal, includes: defining a mapping relationship of the sensor data according to a preset sensor signal mapping table; and generating a corresponding abstract conversion code.

[0109] The generating of the corresponding abstract conversion code includes: converting the sensor data into abstract data; or converting the abstract data into specific sensor data.

[0110] In some embodiments, defining the mapping relationship of sensor data according to a preset sensor signal mapping table includes: reading the correspondence between each sensor signal and the abstract signal from a mapping table (usually an Excel, JSON or XML file), the mapping table containing information such as field name, data type, scaling factor and offset.

[0111] In some implementations, based on sensor types and data requirements, similar sensor signals (such as speed, position, temperature, etc.) are uniformly mapped to standard abstract signal names so that upper-layer applications can call them consistently.

[0112] In some implementations, complex multi-dimensional data or structured data may be mapped layer by layer into an abstract signal structure through nested mapping or indexing.

[0113] In addition, in some implementations, data may be grouped or filtered according to conditions or rules defined in the mapping table, so as to flexibly select specific signals in different application scenarios.

[0114] Through these methods, automatic mapping of sensor data to abstract signals is achieved, laying the foundation for subsequent code generation and data processing.

[0115] In some implementations, generating the corresponding abstract conversion code includes converting the sensor data into abstract data. First, a mapping relationship between the sensor data and the abstract signal is obtained from a mapping table or a configuration file, and a conversion code rule is defined.

[0116] In some implementations, for simple data types, code may be directly generated to convert sensor data fields into abstract data fields one by one, for example, mapping a specific temperature sensor reading into a standard "temperature" abstract signal.

[0117] In some implementations, for data requiring unit conversion, a scaling factor and an offset are introduced into the conversion code to convert the original data into a physical quantity conforming to the abstract signal format.

[0118] In some implementations, if the sensor data is structured multi-dimensional data or an array, the generated conversion code recursively traverses the structure and maps each sub-element into an abstract data structure.

[0119] Furthermore, in some implementations, filtering and processing logic may be generated according to specific conditions of the mapping table to ensure that only data that meets the conditions is converted and passed to the upper-layer application.

[0120] These implementations ensure that sensor data is accurately and standardizedly converted into abstract data for consistent use in the system.

[0121] In some embodiments, generating the corresponding abstract conversion code includes converting the abstract data into specific sensor data. First, according to the correspondence between the abstract data and the sensor data defined in the mapping table, a code is generated to convert the abstract signal into a specific sensor signal format.

[0122] In some implementations, for simple data types, code is directly generated to map abstract fields to specific sensor fields, for example, converting a standard "temperature" signal to the data format of a specific temperature sensor.

[0123] In some implementations, for abstract signals that require unit conversion, the code is generated by applying the inverse scaling factor and offset to convert the abstract physical quantity to the original sensor unit.

[0124] In some implementations, if the abstract data is a complex structure or array, a nested or indexed mapping code is generated to deconstruct the abstract structure into a specific sensor data structure layer by layer.

[0125] In addition, in some implementations, the generated code may also include conditional logic to dynamically select the appropriate sensor format in different application scenarios, ensuring that the abstract signal can be restored to the specific data format of multiple sensors.

[0126] These methods achieve flexible conversion from abstract data to specific sensor data, facilitating diversified applications of the system.

[0127] For example, refer to Figure 6,The sensor signal mapping table example shows the mapping relationship between sensor signals and abstract signals. The left side of the example lists the path and data type of the abstract signal (such as the specific fields in pilot_perception_msg::msg::VisionObjList), and the right side corresponds to the specific signal name and frame information of the sensor (such as the fields in FVCM_LOCALCANFD_55ms_Frame06). Each row represents a specific mapping relationship, such as mapping the raw data such as object size, position, speed, etc. provided by a sensor to abstract standardized signal fields.

[0128] This mapping table is used to generate conversion code to automatically convert raw sensor data into a unified abstract signal format, allowing the system to process data from different sensors in a consistent manner.

[0129] Continue to refer to Figure 6 , describes the abstract conversion process of sensor data. At the beginning of the process, the mapping relationship between sensor data and abstract signals is defined through the mapping table of sensor signals. Then, the corresponding abstract conversion code is generated by the tool to realize the automatic conversion of sensor data to abstract signals. During the conversion process, the input sensor data is received, mapped into standardized abstract signal data through the abstract conversion logic, and the results are output for use by upper-level applications.

[0130] The entire process realizes the automated mapping and conversion from raw sensor data to abstract signals, ensuring the consistency and ease of use of data in the system.

[0131] The embodiment of the present invention also discloses a data analysis system, referring to Figure 7 The system includes a communication configuration module 1, a data parsing module 2, a data abstraction module 3 and a signal sending module 4.

[0132] The communication configuration module 1 is at least used to obtain the communication mode of the sensor and configure the communication mode based on a preset configuration file. Specifically, the communication configuration module 1 first reads the communication mode of the sensor and its related parameters from a preset configuration file (such as JSON or XML format), and the configuration file includes information such as sensor name, data type, communication protocol (such as CAN, Ethernet, SPI, IPCF, etc.), IP address, port number, baud rate, channel path, etc.; then, the communication configuration module 1 sets the corresponding communication interface according to the read communication mode, for example, if the communication mode is CAN, the baud rate and node ID of the CAN bus are configured; if it is Ethernet communication, the IP address and port are configured; if it is SPI, the clock frequency and working mode of the SPI are set; for cross-process communication (IPCF), the parameters of the shared memory or message queue are configured. The communication configuration module 1 can also create an independent working thread to communicate with each sensor to ensure the real-time data reception and the stability of communication. These implementation methods realize the flexible configuration of multiple communication protocols to meet the data transmission requirements of different sensors.

[0133] The data parsing module 2 is at least used to determine the data parsing mode according to the communication mode, and parse the sensor data according to the data parsing mode to obtain the parsed sensor data. Specifically, the data parsing module 2 selects a suitable parsing method according to the configured communication mode. For example, if the communication mode is CAN, the ARXML file is loaded and the corresponding parsing code is called to parse the CAN message into the original physical signal value; if the communication mode is Ethernet, the data parsing module 2 will use protobuf or JSON and other deserialization methods to parse the received data stream into a structured data object; for SPI communication, the data parsing module 2 will parse the data field of each frame according to the frame structure definition and convert it into a specific physical signal; if it is IPCF communication, the data parsing module 2 will read the data from the shared memory or message queue and parse it according to the preset format. In addition, the data parsing module 2 can also perform operations such as unit conversion, range checking or error detection to ensure the integrity and accuracy of the data. These parsing methods ensure that the data parsing module 2 is compatible with a variety of communication protocols and data formats, so that the system can effectively process data from different sensors.

[0134] The data abstraction module 3 is at least used to map the parsed sensor data into a sensor abstract signal. Specifically, the data abstraction module 3 maps the parsed sensor data into a standardized abstract signal through a mapping table or a configuration file, for example, converting specific sensor signals such as temperature and speed into a unified "temperature" or "velocity" abstract signal; for a multi-level data structure, the data abstraction module 3 can recursively map substructures at each level to convert complex sensor data into an abstract data object; when unit conversion or format conversion is required, the data abstraction module 3 will apply a scaling factor and an offset to convert the raw data into a standard unit of the abstract signal; in addition, for specific application requirements, the data abstraction module 3 can merge the data of multiple sensors into a single abstract signal, or filter and process specific signals according to conditions to ensure that the output abstract signal meets the requirements of the upper-level application. In these ways, the data abstraction module 3 unifies multiple sensor data into a standard format, which is convenient for consistent processing by the upper-level system.

[0135] The signal sending module 4 is at least used to send the sensor abstract signal. Specifically, the signal sending module 4 can publish the abstract signal in the form of a ROS message, allowing other ROS nodes to subscribe and use it, which is suitable for applications in the ROS environment; in cross-domain or remote communication scenarios, the signal sending module 4 can transmit the abstract signal to other systems via DDS or Ethernet, using a publish-subscribe mode or direct network transmission to ensure the real-time nature of the signal; for IPC communication, the signal sending module 4 can write the abstract signal to a shared memory or message queue for reading by other processes on the same device; in addition, the signal sending module 4 can also serialize the abstract signal into JSON, protobuf and other formats according to application requirements, and output it through an API or file interface for integration with third-party systems. These sending methods ensure that abstract signals can be transmitted on different platforms and environments to meet diverse communication needs.

[0136] An application scenario of a data analysis system according to an embodiment of the present invention is as follows Figure 8As shown, the sensor abstraction layer is any one of the data parsing systems described in the embodiments of the present invention. ECU1 receives data from multiple sensors (such as sensor 1 to sensor N) through multiple communication protocols such as CAN, IPCF, ETH or SPI, and sends these data to the sensor abstraction layer in ECU2. The sensor abstraction layer parses and standardizes the sensor data into abstract signals, and publishes these abstract signals as different messages (such as lane line messages, control information, etc.) through the data distribution service / communication middleware for application modules on ECU2 (such as application 1 to application N) to subscribe and use. In addition, the parsed physical information message can also be transmitted to the application modules of other ECUs (such as ECU3) through the communication middleware to achieve cross-domain information sharing and system integration. This architecture ensures unified processing and flexible distribution of multi-sensor data, and supports complex multi-module application scenarios.

[0137] The embodiment of the present invention also discloses a readable storage medium.

[0138] A readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a data analysis method described in any one of the above embodiments.

[0139] It can be understood that computer-readable storage media may include: any entity or device capable of carrying a computer program, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc. A computer program includes a computer program code. The computer program code may be in source code form, object code form, an executable file, or some intermediate form, etc. A computer-readable storage medium may include: any entity or device capable of carrying a computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0140] In certain embodiments of the present invention, the electronic device may include a controller or a processor, and the controller is a single-chip microcomputer chip that integrates a processor, a memory, a communication module, etc. The processor may refer to a processor included in the controller. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.

[0141] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0142] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0143] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data analysis method, characterized in that: include: Acquire a communication mode of the sensor, and configure the communication mode based on a preset configuration file; Determining a data parsing method according to the communication method; Parsing the sensor data according to the data parsing method to obtain parsed sensor data; Mapping the parsed sensor data into a sensor abstract signal; And, sending the sensor abstract signal.

2. The data analysis method according to claim 1, characterized in that: Determining the data parsing method according to the communication method includes: If the communication mode is CAN communication, the parsing mode of the CAN signal is configured according to the preset ARXML file; If the communication mode is not CAN communication, the serialization / deserialization mode is configured according to a preset custom data format.

3. The data analysis method according to claim 2, characterized in that: Determining the data parsing method according to the communication method includes: If the data analysis method is a CAN signal analysis method, the sensor signal is converted into a sensor abstract signal; If the data parsing method is a serialization / deserialization method, the sensor data is serialized or deserialized.

4. The data analysis method according to claim 2, characterized in that: The configuration of the parsing method of the CAN signal according to the preset ARXML file includes: Import CAN ARXML file; Generate DBC files corresponding to each CAN channel; Create CAN data parsing code based on the DBC file.

5. The data analysis method according to claim 4, characterized in that: The step of creating a CAN data parsing code according to the DBC file includes: Input CAN payload data; Parse the CAN payload data to obtain the raw value; The raw value is parsed to obtain the physical value.

6. The data analysis method according to claim 2, characterized in that: The configuration of serialization / deserialization mode according to the preset custom data format includes: Define the data structure through an Excel table with custom data format; Parsing the data structure into a corresponding ARXML file; Generate serialization and deserialization codes corresponding to the ARXML file.

7. The data analysis method according to claim 1, characterized in that: The configuring the communication mode based on a preset configuration file includes: Reading a sensor name from the preset configuration file; Create a corresponding working thread according to the sensor name.

8. The data analysis method according to claim 7, characterized in that: The step of creating a corresponding working thread according to the sensor name includes: Confirming the data parsing method according to the data type in the preset configuration file; The data transmission mode is confirmed according to the trans type in the preset configuration file.

9. The data analysis method according to claim 1, characterized in that: Mapping the parsed sensor data into a sensor abstract signal includes: Defining a mapping relationship of sensor data according to a preset sensor signal mapping table; Generate the corresponding abstract transformation code.

10. The data analysis method according to claim 9, characterized in that: The generating of the corresponding abstract conversion code includes: Convert sensor data into abstract data; Alternatively, convert abstract data into concrete sensor data.

11. A data analysis system, characterized in that: The system comprises: A communication configuration module, at least used to obtain a communication mode of the sensor and configure the communication mode based on a preset configuration file; A data analysis module, at least used to determine a data analysis method according to the communication method, and analyze the sensor data according to the data analysis method to obtain analyzed sensor data; A data abstraction module, at least used to map the parsed sensor data into a sensor abstract signal; The signal sending module is at least used to send the sensor abstract signal.

12. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the data analysis method according to any one of claims 1 to 10 is executed.

Images