A data processing system, a data processing method, and a storage medium

Abstract

The application discloses a data processing system, method and storage medium, and relates to the technical field of data processing, in particular to a data processing system, method and storage medium. The data processing system comprises a sensor abstraction module and a processing module. The sensor abstraction module is used for creating a corresponding processing module for a target sensor based on a target configuration type of the target sensor. The processing module is a data processing module or a control processing module. When the processing module is the data processing module, the data processing module is used for performing first processing on original sensor data of the target sensor to obtain first sensor data. When the processing module is the control processing module, the control processing module is used for interacting with the target sensor based on an original control command for the target sensor, so that the target sensor performs a sensor operation or returns second sensor data. The application can reduce the complexity and maintenance difficulty of a sensor data processing system and improve the processing efficiency of sensor data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, and relates to but is not limited to a data processing system, a data processing method and a storage medium. BACKGROUND

[0002] In today's era of rapid development of science and technology, as the key equipment for obtaining data, the types of sensors are becoming increasingly diverse. However, with the increasing number of sensor types, it also brings new challenges to the sensor data processing capability.

[0003] In the related art, the traditional sensor data processing framework has the problems of complicated process and low efficiency when facing massive and complex sensor data, and cannot meet the actual application requirements. Therefore, developing an efficient, flexible and intelligent data processing framework has become an urgent task for the current development of sensor technology. SUMMARY

[0004] One of the purposes of the present application is to provide a data processing system, method and storage medium, which can reduce the complexity and maintenance difficulty of the sensor data processing system and improve the processing efficiency of the sensor data.

[0005] In order to achieve the above purpose, the technical solution adopted by the present application is as follows:

[0006] In a first aspect, the present application provides a data processing system, which comprises a sensor abstraction module and a processing module. The sensor abstraction module is used to create a corresponding processing module for a target sensor based on the target configuration type of the target sensor. The processing module is a data processing module or a control processing module. In the case of the processing module being a data processing module, the data processing module is used to perform first processing on the original sensor data of the target sensor to obtain first sensor data. In the case of the processing module being a control processing module, the control processing module is used to interact with the target sensor based on the original control command for the target sensor to make the target sensor execute a sensor operation or return second sensor data.

[0007] According to the above technical means, by setting the sensor abstraction module in the data processing system, the sensor can be abstracted into different processing modules by the sensor abstraction module according to different configuration types of the sensor, and then the data processing module can be used to complete the data processing process of the sensor, and the control processing module can be used to complete the control interaction process of the sensor. In this way, by abstracting different configuration types of sensors into different processing modules, various types of sensors and various types of sensor data can be classified and processed separately, so as to reduce the complexity and maintenance difficulty of the sensor data processing system and improve the processing efficiency of the sensor data.

[0008] In one possible implementation, the sensor abstraction module includes a parsing module and a management module; the parsing module is used to parse the target configuration file to obtain the target configuration type of the target sensor; the management module is used to obtain the target dynamic link library corresponding to the target sensor; if the target configuration type is a data processing type, a corresponding data processing module is created for the target sensor based on the target dynamic link library; if the target configuration type is a control interaction type, a corresponding control processing module is created for the target sensor based on the target dynamic link library.

[0009] According to the above technical approach, by providing a sensor abstraction module comprising a parsing module and a management module, the parsing module can be used to parse the configuration file to obtain the sensor's configuration type. The management module can then abstract the sensor into a data processing module if the configuration type is data processing, and into a control processing module if the configuration type is control interaction. In this way, by determining whether the sensor's configuration type is data processing or control interaction, data processing sensors can be abstracted into data processing modules, and control interaction sensors can be abstracted into control processing modules, thereby enabling classification and separate processing of various sensors and their data.

[0010] In one possible implementation, the data processing system further includes a service management module; the service management module is used to: perform maintenance management on the sensor abstraction module, and / or perform maintenance management on the processing module; maintenance management at least includes starting, monitoring, and stopping.

[0011] According to the above technical measures, by providing a service management module within the data processing system, it can be used to maintain and manage the sensor abstraction module and / or processing module, such as starting, monitoring, and stopping it. This unified maintenance and management of the sensor abstraction module and / or processing module improves the reliability and maintainability of the data processing system, ensuring its stable and secure operation and providing a strong guarantee for efficient data processing.

[0012] In one possible embodiment, the service management module is further used to: upon receiving a start instruction for the sensor abstract module, call a start function to start the sensor abstract module; after the sensor abstract module is in a running state, obtain at least one data processing module and at least one control processing module included in the data processing system; add at least one data processing module to a data linked list of the management module to put at least one data processing module in a running state, and add at least one control processing module to a control linked list of the management module to put at least one control processing module in a running state; upon receiving a stop instruction for the sensor abstract module, obtain at least one data processing module in the data linked list and at least one control processing module in the control linked list; remove at least one data processing module from the data linked list to put at least one data processing module in a stopped state, and remove at least one control processing module from the control linked list to put at least one control processing module in a stopped state; after at least one data processing module and at least one control processing module are both in a stopped state, call a stop function to stop the sensor abstract module.

[0013] According to the above technical means, after calling the start function to start the sensor abstraction module, the data processing module and the control processing module are added to the corresponding linked list container, thereby achieving startup management of the sensor abstraction module. After removing the data processing module and the control processing module from the corresponding linked list container, the stop function is called to stop the sensor abstraction module, thereby achieving stop management of the sensor abstraction module. In this way, by adding and removing the data processing module and the control processing module from the linked list container, combined with calling the start function and stop function, the startup and stop management of the sensor abstraction module can be achieved efficiently and flexibly, facilitating developers' debugging and optimization, and improving the adaptability and maintainability of the data processing system.

[0014] In one possible implementation, the first processing includes at least receiving and publishing; the data processing module includes at least a receiving component and a publishing component; the receiving component is used to: receive raw sensor data through a receiving thread; take out any data message class from a data message pool; fill the data message class based on the raw sensor data; store the filled data message class into a target message queue; the publishing component is used to: take out the data message class from the target message queue through a publishing thread; publish the first sensor data in the data message class; and put the data message class back into the data message pool.

[0015] According to the above technical approach, by configuring a data processing module to include at least a receiving component and a publishing component, the receiving component can be used to populate any data message class in the data message pool based on raw sensor data and then store it in a message queue. The publishing component can then publish the first sensor data included in the data message class in the message queue and then return the data message class to the data message pool. In this way, by internally constructing the message pool, the repeated creation of data message classes, which results in duplicate resource construction and destruction, can be avoided. By internally maintaining the message queue, unified operation and orderly processing of data streams can be achieved, thereby improving sensor data processing efficiency and system reliability.

[0016] In one possible implementation, when the target message queue is in exclusive mode, if the message processing time of the publishing component is less than the message sending interval, the number of publishing threads is equal to the number of receiving threads; if the message processing time of the publishing component is greater than the message sending interval, the number of publishing threads is greater than the number of receiving threads.

[0017] According to the above technical means, when the target message queue is in exclusive mode and the message processing time of the publishing component is less than the message sending interval, the consistency and orderliness of data processing can be guaranteed by setting the number of publishing threads equal to the number of receiving threads; when the target message queue is in exclusive mode and the message processing time of the publishing component is greater than the message sending interval, the data delay caused by excessive publishing time can be avoided by setting the number of publishing threads greater than the number of receiving threads, thereby ensuring the continuous release of sensor data.

[0018] In one possible implementation, when the target message queue is in shared mode, if the message processing time of the publishing component is less than the message sending interval, the number of publishing threads is less than the number of receiving threads; if the message processing time of the publishing component is greater than the message sending interval, the number of publishing threads is greater than or equal to the number of receiving threads.

[0019] According to the above technical means, when the target message queue is in shared mode and the message processing time of the publishing component is less than the message sending interval, by setting the number of publishing threads to be less than the number of receiving threads, it is possible to ensure efficient and stable operation of the system while avoiding excessive occupation and waste of system resources; when the target message queue is in shared mode and the message processing time of the publishing component is greater than the message sending interval, by setting the number of publishing threads to be greater than or equal to the number of receiving threads, the data processing speed can be increased, data can be prevented from accumulating in the message queue, and the smooth operation of the system can be ensured.

[0020] In a possible embodiment, when the first processing also includes unpacking, decoding, encoding, and packaging, the data processing module also includes an unpacking component, a decoding component, an encoding component, and a packaging component; the unpacking component is used to: unpack the raw sensor data in the data message class from the receiving component; fill the unpacked raw sensor data into the data message class; the decoding component is used to: decode the unpacked raw sensor data in the data message class from the unpacking component, and fill the decoded raw sensor data into the data message class; the encoding component is used to: encode the decoded raw sensor data in the data message class from the decoding component; fill the encoded raw sensor data into the data message class; the packaging component is used to: package the encoded raw sensor data in the data message class from the encoding component; fill the packaged raw sensor data into the data message class; and pass the data message class to the receiving component.

[0021] According to the above technical approach, by configuring the data processing module to also include an unpacking component, a decoding component, an encoding component, and a packaging component, the unpacking component, the decoding component, the encoding component, and the packaging component can be used to unpack, decode, encode, and package the raw sensor data, respectively, to obtain the first sensor data. This not only improves the processing efficiency and accuracy of sensor data, converts the raw sensor data into a data format that is easier to process and transmit, providing a reliable basis for subsequent intelligent decision-making, but also ensures the integrity and security of the data during transmission and storage.

[0022] In one possible embodiment, the data processing module also includes a recording component and a playback component; the recording component is used to: store the first data in a first file; the first data is one of the raw sensor data, unpacked raw sensor data, decoded raw sensor data, encoded raw sensor data, and packaged raw sensor data; the playback component is used to: upon receiving a playback instruction, send the first data in the first file to the first component of the data processing module, so that the first component replays the first data; the first component is one of the receiving component, unpacking component, decoding component, encoding component, packaging component, and publishing component.

[0023] According to the above technical means, by configuring the data processing module to also include a recording component and a playback component, the recording component and the playback component can be used to record and play back raw sensor data, unpacked raw sensor data, decoded raw sensor data, encoded raw sensor data, and packaged raw sensor data. In this way, by recording and playing back data at any stage in the sensor data stream, the maintainability and scalability of the data processing system can be enhanced, providing a reference for data analysis and troubleshooting. In particular, during system debugging and optimization, it can help developers reproduce the data processing process in specific scenarios to adjust and improve system algorithms and parameters.

[0024] In one possible embodiment, the control processing module is also used to: perform a second processing on the original control command to obtain a first control command; the second processing includes at least subscribing and sending; if the first control command is a first command type, sending an operation execution instruction to the target sensor, so that the target sensor responds to the operation execution instruction and performs the sensor operation corresponding to the operation execution instruction; if the first control command is a second command type, sending a data acquisition instruction to the target sensor, so that the target sensor responds to the data acquisition instruction and returns sensor data corresponding to the data acquisition instruction.

[0025] According to the above technical means, by determining whether a control command is of the first command type or the second command type, the system can interact with the sensor to cause it to perform a corresponding sensor operation if the control command is of the first command type, and to cause it to return corresponding sensor data if the control command is of the second command type. In this way, by using the above classified interaction method to cause the sensor to perform a corresponding sensor operation or return corresponding sensor data, precise control and function invocation of the sensor can be achieved, improving the efficiency and flexibility of the interaction between the system and the sensor, and meeting the diverse needs of different application scenarios.

[0026] In a second aspect, the present invention provides a data processing method, which is applied to a data processing system, the data processing system including: a sensor abstraction module and a processing module; the method includes: creating a corresponding processing module for a target sensor based on a target configuration type of the target sensor through the sensor abstraction module; the processing module is a data processing module or a control processing module; in the case where the processing module is a data processing module, the data processing module performs a first processing on the original sensor data of the target sensor to obtain first sensor data; in the case where the processing module is a control processing module, the control processing module interacts with the target sensor based on the original control command for the target sensor, so that the target sensor performs a sensor operation or returns second sensor data.

[0027] In a third aspect, the present application further provides a storage medium, wherein the storage medium stores a computer program, and the computer program is executed to implement the data processing method.

[0028] In a fourth aspect, the present application further provides a computer program product, comprising a computer program or instructions, and the computer program or instructions are executed by a processor to implement the data processing method.

[0029] It should be noted that the technical effects of the second aspect to the fourth aspect can refer to the detailed description of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 A first optional structure schematic diagram of a data processing system provided by an embodiment of the present application is shown in FIG. 1.

[0031] Figure 2 A second optional structure schematic diagram of a data processing system provided by an embodiment of the present application is shown in FIG. 2.

[0032] Figure 3 A third optional structure schematic diagram of a data processing system provided by an embodiment of the present application is shown in FIG. 3.

[0033] Figure 4 A fourth optional structure schematic diagram of a data processing system provided by an embodiment of the present application is shown in FIG. 4.

[0034] Figure 5 A fifth optional structure schematic diagram of a data processing system provided by an embodiment of the present application is shown in FIG. 5.

[0035] Figure 6 A sixth optional structure schematic diagram of a data processing system provided by an embodiment of the present application is shown in FIG. 6.

[0036] Figure 7 An optional structure schematic diagram of a data abstraction framework in a data processing system provided by an embodiment of the present application is shown in FIG. 7.

[0037] Figure 8 An optional structure schematic diagram of a data module and a control module in a data processing system provided by an embodiment of the present application is shown in FIG. 8.

[0038] Figure 9 An optional schematic diagram of a module composition of a data abstraction framework in a data processing system provided by an embodiment of the present application is shown in FIG. 9.

[0039] Figure 10 An optional flow schematic diagram of a module registration management in a data processing system provided by an embodiment of the present application is shown in FIG. 10.

[0040] Figure 11An optional schematic diagram of internal components of a sensor module in a data processing system provided by an embodiment of the present invention;

[0041] Figure 12 An optional schematic diagram of a data abstraction framework startup process in a data processing system provided by an embodiment of the present invention;

[0042] Figure 13 An optional schematic diagram of a data service flow of a data module in a data processing system provided by an embodiment of the present invention;

[0043] Figure 14 An optional schematic diagram of the data flow of a data module in a data processing system provided by an embodiment of the present invention;

[0044] Figure 15 An optional schematic diagram of a control service flow of a control module in a data processing system provided by an embodiment of the present invention;

[0045] Figure 16 An optional schematic diagram of the data flow of a control module in a data processing system provided by an embodiment of the present invention;

[0046] Figure 17 An optional schematic diagram of a data flow based on a message queue in a data processing system provided by an embodiment of the present invention;

[0047] Figure 18 The present invention provides an optional flowchart of a data processing method. DETAILED DESCRIPTION

[0048] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the specific technical solutions of the application will be further described in detail below with reference to the drawings in the embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0049] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0050] In the following description, the terms "first, second, and third" are used merely as examples to distinguish between different objects and do not represent a specific order or precedence for the objects. It is understood that the specific order or precedence of "first, second, and third" can be interchanged where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are for the purpose of describing embodiments of the present invention only and are not intended to limit the present invention.

[0052] Embodiments of the present invention provide a data processing system, method, storage medium, and computer program product.

[0053] Hereinafter, various embodiments of the data processing system, method, storage medium, and computer program product provided by the embodiments of the present invention will be described.

[0054] In a first aspect, an embodiment of the present invention provides a data processing system.

[0055] refer to Figure 1 As shown, a data processing system 10 is connected to a target sensor 20 via a network. The data processing system 10 includes a sensor abstraction module 101 and a processing module 102. The sensor abstraction module 101 is configured to create a corresponding processing module 102 for the target sensor 20 based on the target configuration type of the target sensor 20. The processing module 102 is either a data processing module 1021 or a control processing module 1022. If the processing module 102 is the data processing module 1021, the data processing module 1021 is configured to perform a first process on the raw sensor data of the target sensor 20 to obtain first sensor data. If the processing module 102 is the control processing module 1022, the control processing module 1022 is configured to interact with the target sensor 20 based on the original control command directed to the target sensor 20, so that the target sensor 20 performs a sensor operation or returns second sensor data.

[0056] The target sensor 20 can be any sensor. The embodiment of the present invention does not limit the specific type of the target sensor 20, which can be determined according to actual conditions.

[0057] The sensor abstraction module 101 is used to create a corresponding processing module 102 for the target sensor 20 based on the target configuration type of the target sensor 20 ; the processing module 102 is a data processing module 1021 or a control processing module 1022 .

[0058] Configuration type refers to a pre-configured identifier or classification used to indicate whether a sensor requires control interaction.

[0059] The target configuration type refers to the configuration type of the target sensor 20. The embodiment of the present invention does not limit the specific content of the target configuration type, and it can be determined according to actual conditions.

[0060] Exemplarily, the target configuration type may include a data processing type or a control interaction type. When a sensor is configured as a data processing type, this means that the sensor's corresponding processing module needs to perform various processing operations on the sensor data, such as unpacking, decoding, encoding, and packaging, to meet specific transmission, storage, or analysis requirements. When a sensor is configured as a control interaction type, this means that the sensor's corresponding processing module focuses more on interacting with the sensor, sending specific commands to cause the sensor to perform corresponding operations or obtaining sensor status information. By clarifying the sensor's configuration type, sensors can be managed and used more effectively, improving the efficiency and reliability of the sensor system.

[0061] The data processing module 1021 is configured to perform a first processing on the raw sensor data of the target sensor 20 to obtain first sensor data.

[0062] The embodiments of the present invention do not limit the specific type, format, content, or quantity of raw sensor data from a sensor; these may be determined based on actual circumstances. For example, raw sensor data from a sensor may be the original, unprocessed data collected by the sensor. It reflects the initial state and characteristics of the measured object and may contain various noise and interference.

[0063] The embodiments of the present invention do not limit the specific content of the first processing, which can be determined based on actual circumstances. For example, the first processing can include processing the raw sensor data into a desired format, removing noise and interference from the raw sensor data, or performing multiple processing and transformations on the raw sensor data.

[0064] In actual applications, depending on the difference between the original sensor data and the expected format, the first processing may include only receiving and publishing, or may include receiving, publishing, and at least one of unpacking, decoding, encoding, and packing.

[0065] The first sensor data refers to data obtained after the original sensor data is first processed. The embodiment of the present invention does not limit the specific type, format, content, and quantity of the first sensor data, which can be determined according to actual conditions.

[0066] It can be understood that the first sensor data and the original sensor data may be the same or different.

[0067] The control processing module 1022 is configured to interact with the target sensor 20 based on the original control command directed to the target sensor 20 , so as to enable the target sensor 20 to perform a sensor operation or return second sensor data.

[0068] The embodiments of the present invention do not limit the specific type, format, content, or number of original control commands for sensors, which may be determined based on actual circumstances. For example, original control commands for sensors may be commands sent directly to the sensor to cause it to perform specific operations or return specific data. Examples include a calibration command that calibrates and returns calibration parameters, a start command that activates the sensor for data acquisition, a parameter setting command that adjusts specific sensor parameters and returns the adjustment results, a calibration command that verifies sensor parameters, and a sleep command that puts the sensor into a sleep state to reduce energy consumption. These original control commands are typically generated by a control system or other device based on actual application requirements and transmitted to the sensor to achieve precise control and management of sensor behavior.

[0069] It should be noted that the original control instruction can be transmitted together with other data or can be transmitted separately.

[0070] A sensor operation refers to an operation corresponding to an original control command. For example, if the original control command is a calibration command for verifying sensor parameters, the sensor operation may be a self-calibration operation; if the original control command is a start command for starting the sensor to collect data, the sensor operation may be a start operation.

[0071] Second sensor data refers to the sensor data corresponding to the original control command. For example, if the original control command is a calibration command that calibrates and returns calibration parameters, the second sensor data may be the calibration parameters. If the original control command is a parameter setting command that adjusts specific sensor parameters and returns the adjustment results, the second sensor data may be the adjustment results, such as adjustment success or adjustment failure.

[0072] An embodiment of the present invention provides a data processing system, which includes: a sensor abstraction module and a processing module; the sensor abstraction module is used to: create a corresponding processing module for the target sensor based on the target configuration type of the target sensor; the processing module is a data processing module or a control processing module; when the processing module is the data processing module, the data processing module is used to: perform a first processing on the original sensor data of the target sensor to obtain first sensor data; when the processing module is the control processing module, the control processing module is used to: interact with the target sensor based on the original control command for the target sensor, so that the target sensor performs a sensor operation or returns second sensor data.

[0073] Simply put, by setting up a sensor abstraction module in the data processing system, sensors can be abstracted into different processing modules based on their different configuration types. The data processing module can then be used to complete the sensor data processing process, while the control processing module can be used to complete the sensor control interaction process. By abstracting sensors of different configuration types into different processing modules, different types of sensors and sensor data can be classified and processed separately, reducing the complexity and maintenance difficulty of the sensor data processing system and improving the efficiency of sensor data processing.

[0074] In one possible implementation, reference Figure 2 As shown, the sensor abstraction module 101 includes a parsing module 1011 and a management module 1012. The parsing module 1011 is used to parse the target configuration file to obtain the target configuration type of the target sensor 20. The management module 1012 is used to obtain the target dynamic link library corresponding to the target sensor 20. If the target configuration type is a data processing type, the management module 1012 is used to create a corresponding data processing module for the target sensor 20 based on the target dynamic link library. If the target configuration type is a control interaction type, the management module 1012 is used to create a corresponding control processing module for the target sensor 20 based on the target dynamic link library.

[0075] The parsing module 1011 is configured to parse the target configuration file to obtain the target configuration type of the target sensor 20 .

[0076] The target configuration file is a pre-configured configuration file for storing the target configuration type of the target sensor 20. The embodiment of the present invention does not limit the specific type and content of the target configuration file, which can be determined according to actual conditions. For example, the target configuration file can be a YAML or JSON file.

[0077] In actual applications, the target configuration type of the target sensor 20 may be determined and stored in advance based on human experience or a specific algorithm or a specific model to obtain a target configuration file.

[0078] It is understandable that the target configuration file may also include other configuration parameters of the target sensor 20, such as configuration status (enabled or disabled), configuration parameters, loading path, etc.

[0079] The management module 1012 is used to obtain the target dynamic link library corresponding to the target sensor 20; if the target configuration type is a data processing type, a corresponding data processing module is created for the target sensor 20 based on the target dynamic link library; if the target configuration type is a control interaction type, a corresponding control processing module is created for the target sensor 20 based on the target dynamic link library.

[0080] The dynamic link library refers to a pre-compiled library file which is dynamically loaded and linked during program running. The embodiment of the present application does not limit the specific type and content of the dynamic link library, which can be determined according to actual conditions. Exemplarily, the dynamic link library can be a library file containing a series of functions, variables and resources which can be called by a program. During running, when the program needs to use a specific function in the dynamic link library, the system will load the library into the memory and link the call of the program with the corresponding function in the library. Meanwhile, the dynamic link library can also be conveniently upgraded and updated without recompilation.

[0081] The target dynamic link library refers to the dynamic link library corresponding to the target sensor 20.

[0082] The data processing type is used to represent that the processing module corresponding to the sensor needs to perform data processing operation on the sensor data; and the control interaction type is used to represent that the processing module corresponding to the sensor needs to perform control interaction with the sensor to make the sensor execute a specific operation or return specific data. The embodiment of the present application does not limit the specific form of the data processing type and the control interaction type, which can be determined according to actual conditions. Exemplarily, "0" can be used to represent the data processing type and "1" can be used to represent the control interaction type.

[0083] Briefly, by setting the sensor abstraction module to include the analysis module and the management module, the analysis module can be used to analyze the configuration file to obtain the configuration type of the sensor, and the management module can be used to abstract the sensor as a data processing module when the configuration type is the data processing type and as a control processing module when the configuration type is the control interaction type. In this way, by judging whether the configuration type of the sensor is the data processing type or the control interaction type, the sensor of the data processing type can be abstracted as the data processing module and the sensor of the control interaction type can be abstracted as the control processing module, so as to realize classification and separate processing of various sensors and various sensor data.

[0084] In a possible implementation, referring to the content shown in Figure 3 The service management module 103 is configured to: maintain and manage the sensor abstraction module 101 and / or the processing module 102; and the maintenance and management at least includes starting, monitoring and stopping.

[0085] The service management module 103 is configured to: maintain and manage the sensor abstraction module 101 and / or the processing module 102. The maintenance and management at least includes starting, monitoring and stopping.

[0086] It can be understood that the sensor abstraction module 101 and / or the processing module 102 can realize the above normal functions after being started and in the normal running state; the sensor abstraction module 101 and / or the processing module 102 cannot realize the above normal functions after being stopped and in the stopped state.

[0087] In actual application, the maintenance management can further include other operations. Exemplarily, the maintenance management can further include diagnostic operations (such as running state diagnosis, troubleshooting and positioning, performance screen evaluation and optimization), configuration management operations (such as management of configuration parameters such as server address, port number, database connection parameter, etc.), security management operations (such as access control, data encryption, identity authentication, etc.), backup and recovery, etc.

[0088] Briefly, by setting the service management module in the data processing system, the service management module can be used to maintain and manage the sensor abstraction module and / or the processing module, such as starting, monitoring, stopping, etc. In this way, by uniformly maintaining and managing the sensor abstraction module and / or the processing module, the reliability and maintainability of the data processing system can be improved, the stable operation and safe operation of the data processing system can be ensured, and a strong guarantee is provided for the system to efficiently process data.

[0089] In a possible implementation, the service management module 103 is further configured to: in a case where a starting instruction for the sensor abstraction module 101 is received, call a starting function to start the sensor abstraction module 101; after the sensor abstraction module 101 is in the running state, obtain at least one data processing module and at least one control processing module included in the data processing system 10; add the at least one data processing module to a data linked list of the management module 1012, so that the at least one data processing module is in the running state, and add the at least one control processing module to a control linked list of the management module 1012, so that the at least one control processing module is in the running state; in a case where a stopping instruction for the sensor abstraction module 101 is received, obtain at least one data processing module in the data linked list and at least one control processing module in the control linked list; remove the at least one data processing module from the data linked list, so that the at least one data processing module is in the stopped state, and remove the at least one control processing module from the control linked list, so that the at least one control processing module is in the stopped state; after the at least one data processing module and the at least one control processing module are in the stopped state, call a stopping function to stop the sensor abstraction module 101.

[0090] The start function and the stop function refer to functions for starting and stopping the sensor abstraction module 101. The embodiment of the present invention does not limit the specific content of the start function and the stop function, which can be determined according to actual conditions.

[0091] The data linked list and the control linked list refer to data structures for storing data processing modules and control processing modules, respectively. The embodiment of the present invention does not limit the specific type and size of the data linked list and the control linked list, which can be determined according to actual conditions.

[0092] Simply put, after calling the start function to start the sensor abstraction module, the data processing module and the control processing module are added to the corresponding linked list container to implement startup management of the sensor abstraction module. After removing the data processing module and the control processing module from the corresponding linked list container, the stop function is called to stop the sensor abstraction module to implement stop management of the sensor abstraction module. In this way, by adding and removing the data processing module and the control processing module from the linked list container, combined with calling the start and stop functions, the startup and stop management of the sensor abstraction module can be implemented efficiently and flexibly, facilitating developers' debugging and optimization, and improving the adaptability and maintainability of the data processing system.

[0093] In a possible implementation, the service management module 103 is further configured to:

[0094] When a start instruction for the data processing module 1021 is received, the data processing module 1021 is added to the data linked list of the management module 1012, so that the data processing module 1021 is in a running state; when a stop instruction for the data processing module 1021 is received, the data processing module 1021 is removed from the data linked list of the management module 1012, so that the data processing module 1021 is in a stopped state;

[0095] When a start instruction is received for the control processing module 1022, the control processing module 1022 is added to the control linked list of the management module 1012 so that the control processing module 1022 is in a running state; when a stop instruction is received for the control processing module 1022, the control processing module 1022 is removed from the control linked list of the management module 1012 so that the control processing module 1022 is in a stopped state.

[0096] Simply put, starting and stopping data processing modules or control processing modules by adding and removing them from linked list containers not only avoids unnecessary resource usage but also provides strong guarantees for system flexibility and scalability, enabling the system to dynamically adjust the operating status of modules based on different needs and adapt to various complex application scenarios. Furthermore, the characteristics of linked list containers make addition and removal operations relatively efficient, without significantly impacting system performance.

[0097] In a possible implementation manner, the first processing includes at least receiving and publishing; Figure 4 As shown in the content, the data processing module 1021 includes at least a receiving component 10211 and a publishing component 10212; the receiving component 10211 is used to: receive the original sensor data through a receiving thread; take out any data message class from the data message pool; fill the data message class based on the original sensor data; store the filled data message class into the target message queue; the publishing component 10212 is used to: take out the data message class from the target message queue through a publishing thread; publish the first sensor data in the data message class; and put the data message class back into the data message pool.

[0098] It is understandable that the data processing module 1021 may also include other components, such as at least one of a depacketizing component, a decoding component, an encoding component, and a packaging component, which is not limited in this embodiment of the present invention.

[0099] The data message class refers to the message class used to store sensor data.

[0100] The data message pool refers to the message pool used to store and manage data messages.

[0101] It should be noted that, in the embodiment of the present invention, a data message pool is maintained for each data processing module, that is, each data processing module has its own dedicated data message pool to store and manage sensor data related to the module.

[0102] It can be understood that a data message pool is maintained for each data processing module. First, it can improve the processing efficiency of sensor data. Each module can obtain the required data from its own message pool without searching in the message collection of the entire system, thereby reducing data search time and resource consumption; secondly, it can enhance the scalability of the system. When a new data processing module needs to be added, a new data message pool can be created for it without affecting the existing modules; thirdly, it helps to achieve data isolation and security. The data message pools of different modules are isolated from each other, and the data can be prevented from being modified or leaked by mistake by controlling the access rights to the data.

[0103] The target message queue refers to a message queue used to distribute and process data messages. The embodiment of the present invention does not limit the specific type and capacity of the target message queue, which can be determined according to actual conditions.

[0104] Understandably, target message queues typically have specific rules and processing mechanisms to ensure that data messages are correctly distributed and processed. In practice, you can configure the target message queue's capacity, priority, message processing method, and other parameters based on your business needs to achieve efficient data transmission and processing.

[0105] It should be noted that the target message queue can be in exclusive mode or shared mode.

[0106] Simply put, by configuring the data processing module to include at least a receiving component and a publishing component, the receiving component can be used to populate any data message class in the data message pool based on raw sensor data and then store it in a message queue. The publishing component can then publish the first sensor data included in the data message class in the message queue and return the data message class to the data message pool. This internally constructed message pool avoids the repeated creation of data message classes, which results in duplicate resource construction and destruction. Internally maintaining the message queue allows for unified operation and orderly processing of data streams, thereby improving sensor data processing efficiency and system reliability.

[0107] In one embodiment, when the target message queue is in exclusive mode, if the message processing time of the publishing component 10212 is less than the message sending interval, the number of the publishing threads is equal to the number of the receiving threads; if the message processing time of the publishing component 10212 is greater than the message sending interval, the number of the publishing threads is greater than the number of the receiving threads.

[0108] It can be understood that in exclusive mode, the target message queue is only accessed and used by a certain module. Exclusive mode can ensure that the module has full control over the message queue and avoid conflicts with other modules. It is especially suitable for scenarios with high requirements on message processing order and timeliness.

[0109] The message processing time refers to the time it takes for the publishing component 10212 to process a single data message class. In this embodiment of the present invention, the message processing time refers to the difference between the time when the publishing component 10212 starts to retrieve the data message class from the target message queue and the time when the data message class is returned to the data message pool.

[0110] The message sending interval refers to a pre-configured time interval between two consecutive messages sent by the publishing component 10212. The embodiment of the present invention does not limit the specific value of the message sending interval, which can be determined according to actual conditions.

[0111] Simply put, when the target message queue is in exclusive mode and the message processing time of the publishing component is less than the message sending interval, by setting the number of publishing threads equal to the number of receiving threads, the consistency and orderliness of data processing can be guaranteed; when the target message queue is in exclusive mode and the message processing time of the publishing component is greater than the message sending interval, by setting the number of publishing threads greater than the number of receiving threads, data delays caused by excessive publishing time can be avoided, ensuring the continuous release of sensor data.

[0112] In one embodiment, when the target message queue is in shared mode, if the message processing time of the publishing component 10212 is less than the message sending interval, the number of publishing threads is less than the number of receiving threads; if the message processing time of the publishing component 10212 is greater than the message sending interval, the number of publishing threads is greater than or equal to the number of receiving threads.

[0113] It can be understood that in the shared mode, the target message queue can be accessed and used by multiple modules in accordance with specific rules. The shared mode can improve the utilization of the message queue and realize resource sharing and collaborative work.

[0114] Simply put, when the target message queue is in shared mode and the message processing time of the publishing component is less than the message sending interval, by setting the number of publishing threads to be less than the number of receiving threads, it is possible to ensure efficient and stable operation of the system while avoiding excessive occupation and waste of system resources; when the target message queue is in shared mode and the message processing time of the publishing component is greater than the message sending interval, by setting the number of publishing threads to be greater than or equal to the number of receiving threads, it is possible to increase the data processing speed, prevent data from accumulating in the message queue, and ensure smooth operation of the system.

[0115] In a possible implementation manner, when the first processing further includes unpacking, decoding, encoding, and packaging, refer to Figure 5 As shown in the content, the data processing module 1021 also includes an unpacking component 10213, a decoding component 10214, an encoding component 10215, and a packaging component 10216;

[0116] The unpacking component 10213 is used to: unpack the raw sensor data in the data message class from the receiving component 10211; and fill the unpacked raw sensor data into the data message class;

[0117] The decoding component 10214 is used to decode the unpacked raw sensor data in the data message class from the unpacking component 10213, and fill the decoded raw sensor data into the data message class;

[0118] The encoding component 10215 is configured to encode the decoded raw sensor data in the data message class from the decoding component 10214, and fill the encoded raw sensor data into the data message class.

[0119] The packaging component 10216 is configured to package the encoded raw sensor data in the data message class from the encoding component 10215, fill the packaged raw sensor data into the data message class, and deliver the data message class to the receiving component 10211.

[0120] The embodiment of the present application does not limit the unpacking manner, decoding manner, encoding manner, and packaging manner of the unpacking component 10213, the decoding component 10214, the encoding component 10215, and the packaging component 10216, and can be determined according to actual conditions.

[0121] It should be noted that after the data message class is filled by any of the unpacking component 10213, the decoding component 10214, the encoding component 10215, and the packaging component 10216, it finally needs to be delivered to the receiving component 10211, and the filled data message class is put into the data message pool and stored in the target message queue by the receiving component 1021.

[0122] In short, by setting the data processing module to further include the unpacking component, the decoding component, the encoding component, and the packaging component, the raw sensor data can be unpacked, decoded, encoded, and packaged by the unpacking component, the decoding component, the encoding component, and the packaging component, respectively, to obtain the first sensor data. In this way, not only can the processing efficiency and accuracy of the sensor data be improved, the raw sensor data can be converted into a data format that is easier to process and transmit, providing a reliable basis for subsequent intelligent decision-making, but also the integrity and security of the data in the transmission and storage process can be ensured.

[0123] In a possible implementation, reference is made to Figure 6As shown in the content, the data processing module 1021 also includes a recording component 10217 and a playback component 10218; the recording component 10217 is used to: store the first data in the first file; the first data is one of the original sensor data, the unpacked original sensor data, the decoded original sensor data, the encoded original sensor data, and the packaged original sensor data; the playback component 10218 is used to: upon receiving a playback instruction, send the first data in the first file to the first component of the data processing module, so that the first component replays the first data; the first component is one of the receiving component 10211, the unpacking component 10213, the decoding component 10214, the encoding component 10215, the packaging component 10216, and the publishing component 10212.

[0124] The first data refers to sensor data that needs to be recorded and played back, and can be any one of raw sensor data, unpacked raw sensor data, decoded raw sensor data, encoded raw sensor data, and packaged raw sensor data.

[0125] The first component refers to the component that needs to replay the first data, which can be any one of the receiving component 10211, the unpacking component 10213, the decoding component 10214, the encoding component 10215, the packaging component 10216, and the publishing component 10212.

[0126] It can be understood that the first data and the first component correspond to each other. If the first data is raw sensor data, the first component is the receiving component 10211 or the unpacking component 10213; if the first data is unpacked raw sensor data, the first component is the decoding component 10214; if the first data is decoded raw sensor data, the first component is the encoding component 10215; if the first data is encoded raw sensor data, the first component is the packaging component 10216; if the first data is packaged raw sensor data, the first component is the publishing component 10212.

[0127] Simply put, by configuring the data processing module to also include recording and playback components, these components can be used to record and replay raw sensor data, unpacked raw sensor data, decoded raw sensor data, encoded raw sensor data, and packaged raw sensor data. By recording and replaying data at any stage in the sensor data stream, the maintainability and scalability of the data processing system can be enhanced, providing a reference for data analysis and troubleshooting. In particular, during system debugging and optimization, this can help developers reproduce the data processing process in specific scenarios, allowing them to adjust and improve system algorithms and parameters.

[0128] In a possible embodiment, the control processing module 1022 is also used to: perform a second processing on the original control command to obtain a first control command; the second processing includes at least subscribing and sending; if the first control command is a first command type, sending an operation execution instruction to the target sensor 20, so that the target sensor 20 responds to the operation execution instruction and performs the sensor operation corresponding to the operation execution instruction; if the first control command is a second command type, sending a data acquisition instruction to the target sensor 20, so that the target sensor 20 responds to the data acquisition instruction and returns sensor data corresponding to the data acquisition instruction.

[0129] The second processing includes at least subscription and sending; the control processing module 1022 includes at least subscription component and sending component.

[0130] The embodiment of the present invention does not limit the specific content of the second processing, which can be determined according to actual conditions.

[0131] In actual applications, depending on whether the original control command is transmitted along with other data, the second processing may include only subscription and sending, or may include subscription, sending, and at least one of requesting a reply, unpacking, decoding, and encoding. Similarly, the control processing module 1022 may include only subscription and sending components, or may include subscription and sending components, and at least one of requesting a reply, unpacking, decoding, and encoding components.

[0132] For the specific implementation of subscription, sending, request reply, unpacking, decoding, and encoding in the control processing module 1022, please refer to the detailed description of the above-mentioned data processing module 1021, which will not be repeated here.

[0133] The first control command refers to a command obtained after the original control command is subjected to the second processing. The embodiment of the present invention does not limit the specific type, format, content, and quantity of the first control command, which can be determined according to actual conditions.

[0134] The first command type is used to represent a command type that requires the sensor to perform a sensor operation but does not need to return sensor data. The second command type is used to represent a command type that requires the sensor to return sensor data. The embodiments of the present invention do not limit the specific representation of the first and second command types, which can be determined based on actual circumstances. For example, "0" can be used to represent the first command type, and "1" can be used to represent the second command type.

[0135] In actual applications, when the first control command is of the first command type, the operation execution instruction can be determined based on the specific sensor operation carried in the first control command, and the operation execution instruction also carries the specific sensor operation to instruct the target sensor 20 to perform the specific sensor operation; when the first control command is of the second command type, the data acquisition instruction can be determined based on the identifier of the specific sensor data to be returned carried in the first control command, and the data acquisition instruction also carries the identifier of the specific sensor data to be returned to instruct the target sensor 20 to return the specific sensor data.

[0136] Simply put, by determining whether a control command is of the first or second command type, the system can interact with the sensor to cause it to perform a corresponding sensor operation if the control command is of the first command type, and to cause it to return corresponding sensor data if the control command is of the second command type. This categorized interaction method allows the sensor to perform corresponding sensor operations or return corresponding sensor data, enabling precise control and function invocation of the sensor. This improves the efficiency and flexibility of system-sensor interaction and meets the diverse needs of different application scenarios.

[0137] The data processing method provided by the present invention is described below through a detailed embodiment.

[0138] The automotive industry is becoming a leading technology direction in a new round of technological revolution and industrial development, with software-defined cars being the mainstream development approach of the future. Demand for intelligent, secure, and technologically advanced automotive software is increasing, but smart cars rely heavily on sensors. Processing data from multiple onboard sensors still presents technical bottlenecks and security issues, leading to increasing demands for simplified software development and operational reliability. Related technologies do not address multi-sensor management for scenarios where multiple sensors operate in parallel or for differentiated sensor deployment across multiple vehicle models. Their use cases are limited and cannot meet the requirements of multi-vehicle, multi-sensor applications.

[0139] Different vehicle models carry different types, models, and quantities of sensors. Even the same vehicle model needs to be equipped with multiple sensors, such as radar, surround-view cameras, and so on. To solve the problem of multi-sensor data processing in intelligent driving systems, the present invention proposes a vehicle-mounted sensor data abstraction framework method for multiple vehicle models and multiple sensors, so as to improve the processing efficiency and data security of vehicle-mounted sensor data, reduce resource consumption, and thus improve driving safety.

[0140] The overall framework description and functional flow of the present invention are described in detail below with reference to the accompanying drawings.

[0141] For the overall structure diagram of the data abstraction framework, see Figure 7 The overall framework includes an abstract framework module (DAS Framework) 701 and n sensor modules (SensorModule) 702. Each sensor module is individually compiled into a module library and independently registered with the framework for use, allowing for flexible combinations of various sensors. Different vehicle models require different sensor scenarios, and the required sensors can be installed based on the vehicle model. Therefore, the in-vehicle sensor abstract framework method proposed in the present invention can be adapted to different vehicle models.

[0142] The sensor module includes a data module and a control module. Figure 8 As shown in the figure, the data module (DateService) 801 includes a data service (DataService) 8011, a receiving service (RecvServer) 8012, a publishing service (PubServer) 8013, a receiving (Receiver) component 8014, a decoding (Decoder) component 8015, a message transmission (Message) component 8016, a packaging (Packer) component 8017, a publishing (Publisher) component 8018, a first recording (Recorder) component 8019, and a first playback (Player) component. 8020; The control module (CtrlService) 802 includes the control service (CAservice) 8021, the request and reply service (RpcServer) 8022, the subscription service (Subserver) 8023, as well as the subscription (Subscriber) component 8024, the packaging (Encoder) component 8025, the unpacking (Unpacker) component 8026, the request and reply (Rpc) component 8027, the sending (Sendsor) component 8028, the second recording (Recorder) component 8029, and the second playback (Player) component 8030. However, both the data module and the control module are loaded and managed by the service management module (ServiceMgmt).

[0143] The module composition diagram of the data abstraction framework can be found in Figure 9, including parsing module 901, management module 902, service module 903, and service management module 904. The parsing module loads the file function and stores the parsed configuration parameters in the storage container inside the parsing module; the management module has a loading module interface inside to load the module into the internally managed module list; the service module is a class member of the service management module, and the formation of the service module component depends on the internal method implementation of the parsing module and the management module; the service management module contains the parsing module and the management module to create an instance object. When it is necessary to create a service module, it is only necessary to create the service management module. The service management module will pass the configuration file into its own internal parsing module object to obtain the corresponding module information. The service management module will pass the obtained module information into its own internal management module object. The management module object will achieve the purpose of creating the control module and data module by calling its own loading module method. The service management module will place the created data module and control module into the internally maintained linked list container respectively, and realize the module start, run, stop and other functions by calling the start and stop methods inside the service management module.

[0144] Module registration management flow chart see Figure 10 The service component (Service) 1001 starts registering the module service, i.e., sensor module registration, by reading the configuration file (Configuration), and processes the complete data stream from reception to release; the module management module (ModuleMgmt) 1002 starts managing a single sensor module; the service management module (ServiceMgmt) 1003 implements the management of all sensors, including starting / stopping sensors, etc.

[0145] See the diagram of the sensor module internal components. Figure 11 For each sensor module, the data abstraction framework needs to create a receiving component 1102, a decoding component 1103, a messaging component 1104, and a publishing component 1105 through a singleton factory 1101. Each module can be independently compiled into a dynamic link library and registered with the data abstraction framework. The data abstraction framework requires that the sensor module must create a receiving component, a decoding component, a messaging component, and a publishing component. At the same time, the module maintains a singleton factory internally, through which the above component classes are created. The module supports the compilation of dynamic link libraries, which are then registered with the data abstraction framework. For each abstract module, the data abstraction framework distinguishes it through the module identifier (ModuleID) set by the singleton factory.

[0146] 1. The data abstraction framework parses the configuration file (such as yaml or json file) to read the module name, loading path, module type to be loaded, and other parameters of the sensor abstraction module, and stores the parsed information in the management module; each sensor data processing module can be abstracted into an independent module, and can be compiled into an independent dynamic library; the management module creates a data module and a control module based on the parsed information, and loads the dynamic library compiled by the sensor abstraction module into the linked list inside the management module (ServiceMgmt) to meet the needs of different models for different sensors; the management module starts and stops the data module and the control module, thereby realizing the start and stop operations within the module. See the data abstraction framework startup flow chart for details. Figure 12 , including the following steps:

[0147] S1201: The parsing module parses the configuration file and stores the parsed configuration parameters into the internal members of the parsing module.

[0148] S1202: The parsing module transmits the parsed configuration parameters to the management module.

[0149] S1203: The management module determines the module type. Based on the attributes of the configuration parameters, if the module type is a data module, the management module executes S1204; if the module type is a control module, the management module executes S1205.

[0150] S1204: The management module creates a data module and loads the data module into an internal data module linked list.

[0151] S1205: The management module creates a control module and loads the control module into an internal control module linked list.

[0152] S1206: The service management module determines whether the data abstraction service needs to be started. If yes, execute S1207; otherwise, execute S1212.

[0153] S1207: The service management module starts the data abstraction service. When external conditions are met (when the data abstraction service needs to be started), the sub-services are sequentially taken out from the data module chain list and the control module chain list of the management module and started.

[0154] S1208. The service management module determines whether it is necessary to stop some module services. If so, execute S1209; otherwise, execute S1212.

[0155] S1209: The service management module stops some module services. When some module services need to be stopped, the data abstraction framework removes some submodules from the control module list or data module list according to the internal logic, thereby stopping some module services.

[0156] S1210. The service management module determines whether the data abstraction service needs to be stopped. If so, execute S1211; otherwise, execute S1212.

[0157] S1211: The service management module stops the data abstraction service. When the data abstraction service needs to be stopped, the data abstraction framework loads an internal stop function and sequentially retrieves sub-services from the control module list and data module of the management module to stop.

[0158] S1212: Continue running the existing module.

[0159] 2. After the data module has gone through the process of receiving, unpacking, packaging, and publishing, it abstracts the original data into the target type and then publishes it. Figure 13 , including the following steps:

[0160] S1301, determine whether the data module is running, if so, execute S1302, if not, execute S1305.

[0161] S1302: Receive sensor data.

[0162] S1303, parse sensor data. When the data module is running, after successfully receiving the raw data sent by the sensor, the raw data is parsed.

[0163] S1304: Release sensor data. The sensor data is packaged according to the release protocol and the packaged data is finally released within the release module, thus completing the entire data flow of the data module.

[0164] S1305, end.

[0165] The data flow diagram of the data module can be found in Figure 14, the data abstraction framework maintains a message pool (MessagePool) for each individual data module to avoid repeated construction of the Message storage class. The number of messages created by the message pool is read from the configuration file. The basic type of the message pool is a class composed of Message components. After the Publisher component inside the module takes out the message from the message pool and uses it, it will store the used Message storage class back into the MessagePool. The Receiver component will take out the previously used Message from the MessagePool for reuse and put the received original data into the Message again. After the Publisher component takes out the previously used Message from the MessagePool and reuses it, it will put the received original data into the Message again. component, thereby publishing the data; when the data module receives sensor data and abnormal information occurs, the data abstraction framework will directly skip the complex processes such as unpacking and packaging, jump directly into the publishing process, print the abnormal information, and store the abnormal information in the log; for the data module's re-injection mode, the data abstraction framework will call the Recorder component during the receiving and publishing processes, and store the received sensor data and the sent target data into a dat file in a certain format. When running offline, the data abstraction module will parse the previously recorded dat file through the Player component, and re-inject the previous data to the existing component module through the Player component, thereby sending the previous original data or target data in reverse.

[0166] 3. After the control module parses the subscribed control command, it operates according to the parsing protocol. If the type is a control sensor, it publishes the control sensor parameters to the sensor module. If it is a state acquisition, it publishes the corresponding state acquisition command to the corresponding module and waits for feedback from the corresponding module. Figure 15 , including the following steps:

[0167] S1501. Determine whether the control module is running. If so, execute S1502; otherwise, execute S1507.

[0168] S1502: Subscribe to receive control commands. When the control module is running, subscribe to the corresponding control commands.

[0169] S1503: Parse the control command: When the subscription control command is successfully received, parse the subscription command.

[0170] S1504. Determine the control command type. If it is sensor control, execute S1505. If it is status acquisition, execute S1506. If it is not supported, return to S1501 and continue to wait for the next control command.

[0171] S1505: Send sensor control parameters. If the command type after parsing is determined to be sensor control, the sensor control parameters are sent to the sensor module.

[0172] S1506: Send a status acquisition command and wait for a status reply: If the command type after parsing is determined to be a status acquisition command, send a status acquisition command to the corresponding module and wait for the corresponding module to reply a status.

[0173] S1507, end.

[0174] The data flow diagram of the control module can be found in Figure 16 The data abstraction framework maintains a MessagePool for each individual control module. The number of messages created in this message pool is read from the configuration file. When the Sender component finishes using a Message, it returns the used Message class to the message pool, awaiting the next use by the Subscriber or RpcServer component. Message is a component class that contains the Packet data to be published. The data to be packaged is generally stored in the Message class, which passes the Packet data from the Receiver component to the Publisher component.

[0175] 4. A message queue (MessageQueue) is built for different modules within the data abstraction framework. This message queue can be exclusive to a single module or shared by multiple modules. When the message queue is created, if it is used by a single data module alone, the receiving thread pushes the received data into the queue, and the sending thread retrieves the received data from the queue. If the publishing thread takes a short time to group packages, and the sending thread and the receiving thread maintain an equal number (the default is 1), the order of sending data is ensured to be consistent. The sensor receiving component will fill the received data into the Message class, the data abstraction framework will push the Message class into the message queue, the sensor publishing component will retrieve the data from the message queue, fill the information in the Message class into the sending target data with reference to the serialization protocol, and finally send it out through the publishing framework.

[0176] The receiving thread and the publishing thread of the data abstraction framework are independently operated, the receiving thread pushes the data into the message queue after receiving the data, and the publishing thread takes out from the message queue. When the message queue is unique for a single module, if the processing time of the sending thread is short, the data abstraction framework creates the same number of threads as the receiving thread; if the packaging time of the sending thread is long, the sensor sending interval period received by the receiving thread is much higher, at this time, the data abstraction framework supports creating multiple sending threads, the sending periods of the multiple threads are staggered, and the data is taken out from the queue in turn for packaging, so as to maintain the stability of the sending period and the consistency of the data sequence. When the packaging time of the sending thread is short, the sensor sending interval period received by the receiving thread is much lower, the data abstraction framework supports multiple sensor modules using the same message queue. When the message queue is shared by multiple modules, if the processing time of the sending data is short, the received data of the receiving threads of multiple modules can be put into the same queue, the data abstraction framework supports reducing the number of sending threads, and the data abstraction framework creates fewer sending threads than the number of receiving threads, so as to realize the sensor data publishing, reduce the number of threads, reduce the overall resource usage of the data abstraction framework, and achieve the purpose of reducing the overall resource consumption.

[0177] The data flow implementation diagram based on the message queue is shown in Figure 17 The sensor data from the sensor 1701 is transmitted to the publishing component 17014 after being processed by the receiving component 17011, the decoding component 17012 and the packaging component 17013, three messages in the message queue are processed by three publishing threads, the three publishing threads are staggered in the sending interval period, and finally published to the data bus 1703, so as to solve the problem that the packaging time is too long to cause the sending interval period to exceed the sensor sending interval period. Similarly, the sensor data from the sensor 1702 is transmitted to the publishing component 17024 after being processed by the receiving component 17021, the decoding component 17022 and the packaging component 17023, and finally published to the data bus 1703.

[0178] The car-mounted sensor data abstraction framework and method have the following beneficial effects:

[0179] a. Flexible and dynamic loading of sensor modules. The data abstraction framework creates corresponding sensor modules by reading configuration files (not limited to JSON or YAML files), loads the dynamic link library compiled by the sensor module, loads the sensor abstraction module by using the dynamic loading method, so as to realize the dynamic business demand of various vehicle models and various sensors; the management module of the data abstraction framework manages the running state of the sensor abstraction module, and realizes the start, stop, removal, loading and the like of the sensor module.

[0180] b. A message pool is built within the data abstraction framework to avoid repeated creation of Message components, which leads to repeated construction and destruction of resources.

[0181] c. Distributed processing of sensor module data. The data abstraction framework maintains a message queue for each sensor abstraction module. The sensor abstraction module pushes received data into the message queue. When the sensor processing module publishes data, it retrieves the data from the message queue and serializes it. By maintaining the message queue internally, the data abstraction framework can allow multiple sensor modules to share the message queue or allow each sensor module to have its own queue, thereby achieving unified operation of the data flow.

[0182] d. When multiple sensors are deployed and sensor data processing takes too long, or module packaging takes a long time, multiple sending threads are created through the multi-threaded number configured in the data abstraction framework. Data is taken out from the message queue in sequence and packaged and published at intervals, completing the packaging and serialization operations together, thereby achieving the goal of optimizing packaging time and stably publishing complex data at a regular period.

[0183] 4. When targeting multiple vehicle models, developers only need to refer to the data abstraction framework design to complete the construction of a single module, compile it into a dynamic link library, and load and run the module by modifying the configuration file parameters. This approach allows developers to focus on developing the sensor abstraction module, thereby shortening development time and cycle time, improving development efficiency, and effectively enhancing software performance. Furthermore, by modifying the configuration file parameters, the business needs of different sensors for different vehicle models can be met.

[0184] In a second aspect, an embodiment of the present invention provides a data processing method, which is applied to a data processing system. The data processing system includes: a sensor abstraction module and a processing module.

[0185] The connection method of the sensor abstraction module and the processing module as well as the hardware structure and software structure can refer to the detailed description of the first aspect above, which will not be repeated here.

[0186] refer to Figure 18 The vehicle control method includes but is not limited to the following: Figure 18 S1801 to S1803 shown.

[0187] S1801: Create a corresponding processing module for the target sensor based on the target configuration type of the target sensor through the sensor abstraction module.

[0188] The processing module is a data processing module or a control processing module.

[0189] The implementation process of S1801 can refer to the description of the sensor abstraction module in the first aspect above, and will not be repeated here.

[0190] S1802, in the case where the processing module is the data processing module, performing first processing on the raw sensor data of the target sensor by the data processing module to obtain first sensor data.

[0191] The implementation process of S1802 can refer to the description of the data processing module in the first aspect above, and will not be repeated here.

[0192] S1803, in the case where the processing module is the control processing module, interacting with the target sensor based on the raw control command for the target sensor by the control processing module to make the target sensor perform a sensor operation or return second sensor data.

[0193] The implementation process of S1803 can refer to the description of the control processing module in the first aspect above, and will not be repeated here.

[0194] In short, by adopting the sensor abstraction module to abstract the sensor into different processing modules according to different configuration types of the sensor, the data processing module can be used to complete the data processing process of the sensor, and the control processing module can be used to complete the control interaction process of the sensor. In this way, by abstracting different configuration types of the sensor into different processing modules, various types of sensors and various types of sensor data can be classified and processed separately, so as to reduce the complexity and maintenance difficulty of the sensor data processing system and improve the processing efficiency of the sensor data.

[0195] In a third aspect, the present application further provides a storage medium, wherein the storage medium stores a computer program, and the computer program stored on the storage medium is executed to implement the steps of the data processing method provided in the embodiments of the present application.

[0196] In a fourth aspect, the present application further provides a computer program product comprising a computer program or instructions, and the computer program or instructions are executed by a processor to implement the steps of the above data processing method.

[0197] It should be noted that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

[0198] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the embodiments of the present invention mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0199] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0200] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0201] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0202] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0203] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware, the foregoing program can be stored in a computer readable storage medium, and the program executes the steps of the method embodiments when executed; and the foregoing storage medium includes a mobile storage device, a read only memory (ROM), a magnetic disc or an optical disc, and various storage medium capable of storing program codes.

[0204] Alternatively, the integrated unit of the present application can be stored in a computer readable storage medium if it is realized in the form of a software function module and sold or used as an independent product. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the data processing method described in the embodiments of the present application. The foregoing storage medium includes a mobile storage device, a ROM, a magnetic disc or an optical disc, and various storage medium capable of storing program codes.

[0205] The above is only an embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A data processing system, characterized in that: The data processing system includes: a sensor abstraction module and a processing module; The sensor abstraction module is used to: create a corresponding processing module for the target sensor based on the target configuration type of the target sensor; the processing module is a data processing module or a control processing module; In the case where the processing module is the data processing module, the data processing module is configured to: perform first processing on the raw sensor data of the target sensor to obtain first sensor data; In a case where the processing module is the control processing module, the control processing module is configured to: interact with the target sensor based on the original control command for the target sensor, so that the target sensor performs a sensor operation or returns second sensor data; Wherein, the sensor abstract module includes an analysis module and a management module; The parsing module is used to: parse the target configuration file to obtain the target configuration type of the target sensor; The management module is used to: obtain a target dynamic link library corresponding to the target sensor; if the target configuration type is a data processing type, create a corresponding data processing module for the target sensor based on the target dynamic link library; if the target configuration type is a control interaction type, create a corresponding control processing module for the target sensor based on the target dynamic link library; the target dynamic link library refers to a dynamic link library corresponding to the target sensor; the dynamic link library refers to a pre-compiled library file that is dynamically loaded and linked when the program is running; Wherein, the first processing includes at least receiving and publishing; the data processing module includes at least a receiving component and a publishing component; The receiving component is used to: receive the raw sensor data through a receiving thread; take out any data message class from a data message pool; fill the data message class based on the raw sensor data; and store the filled data message class into a target message queue; The publishing component is used to: retrieve the data message class from the target message queue through a publishing thread; publish the first sensor data in the data message class; and put the data message class back into the data message pool.

2. The data processing system according to claim 1, wherein: The data processing system further includes a service management module; The service management module is used to: perform maintenance management on the sensor abstraction module and / or perform maintenance management on the processing module; the maintenance management at least includes starting, monitoring, and stopping.

3. The data processing system according to claim 2, wherein: The service management module is also used to: Upon receiving a startup instruction for the sensor abstraction module, calling a startup function to start the sensor abstraction module; after the sensor abstraction module is in a running state, obtaining at least one data processing module and at least one control processing module included in the data processing system; Adding the at least one data processing module to the data linked list of the management module so that the at least one data processing module is in a running state, and adding the at least one control processing module to the control linked list of the management module so that the at least one control processing module is in a running state; the control linked list refers to a data structure for controlling the processing module; the data linked list refers to a data structure for storing the data processing module; Upon receiving a stop instruction for the sensor abstraction module, acquiring at least one data processing module in the data linked list and at least one control processing module in the control linked list; The at least one data processing module is removed from the data linked list to put the at least one data processing module into a stopped state, and the at least one control processing module is removed from the control linked list to put the at least one control processing module into a stopped state; after the at least one data processing module and the at least one control processing module are both in the stopped state, a stop function is called to stop the sensor abstract module.

4. The data processing system according to claim 1, wherein: When the target message queue is in exclusive mode, if the message processing time of the publishing component is less than the message sending interval, the number of publishing threads is equal to the number of receiving threads; if the message processing time of the publishing component is greater than the message sending interval, the number of publishing threads is greater than the number of receiving threads; the exclusive mode means that the target message queue is only accessed and used by a certain module alone.

5. The data processing system according to claim 1, wherein: When the target message queue is in shared mode, if the message processing time of the publishing component is less than the message sending interval, the number of publishing threads is less than the number of receiving threads; if the message processing time of the publishing component is greater than the message sending interval, the number of publishing threads is greater than or equal to the number of receiving threads; the shared mode means that the target message queue can be accessed and used by multiple modules according to specific rules.

6. The data processing system according to claim 1, wherein: In the case where the first processing further includes unpacking, decoding, encoding, and packing, the data processing module further includes an unpacking component, a decoding component, an encoding component, and a packing component; The unpacking component is used to: unpack the raw sensor data in the data message class from the receiving component; and fill the unpacked raw sensor data into the data message class; The decoding component is used to: decode the unpacked raw sensor data in the data message class from the unpacking component, and fill the decoded raw sensor data into the data message class; The encoding component is used to: encode the decoded raw sensor data in the data message class from the decoding component; and fill the encoded raw sensor data into the data message class; The packaging component is used to: package the encoded raw sensor data in the data message class from the encoding component; fill the packaged raw sensor data into the data message class; and pass the data message class to the receiving component.

7. The data processing system according to claim 6, wherein: The data processing module also includes a recording component and a playback component; The recording component is configured to store first data in a first file; the first data being one of the raw sensor data, the unpacked raw sensor data, the decoded raw sensor data, the encoded raw sensor data, and the packaged raw sensor data; The playback component is configured to: upon receiving a playback instruction, send the first data in the first file to the first component of the data processing module, so that the first component plays back the first data; The first component is one of the receiving component, the unpacking component, the decoding component, the encoding component, the packaging component, and the publishing component.

8. The data processing system according to claim 1, wherein: The control processing module is further configured to: Performing a second process on the original control command to obtain a first control command; the second process at least includes subscribing and sending; If the first control command is of the first command type, sending an operation execution instruction to the target sensor, so that the target sensor responds to the operation execution instruction and performs a sensor operation corresponding to the operation execution instruction; If the first control command is of the second command type, a data acquisition instruction is sent to the target sensor, so that the target sensor responds to the data acquisition instruction and returns sensor data corresponding to the data acquisition instruction.

9. A data processing method, characterized in that: The method is applied to the data processing system according to claim 1, wherein the data processing system comprises: a sensor abstraction module and a processing module; the method comprises: Creating a corresponding processing module for the target sensor based on the target configuration type of the target sensor through the sensor abstract module; the processing module is a data processing module or a control processing module; In a case where the processing module is the data processing module, performing first processing on the raw sensor data of the target sensor by the data processing module to obtain first sensor data; In the case where the processing module is the control processing module, the control processing module interacts with the target sensor based on the original control command for the target sensor, so that the target sensor performs a sensor operation or returns second sensor data.

10. A computer-readable storage medium, characterized in that The storage medium stores a computer program or instruction, and when the computer program or instruction is executed by the processor, the method according to claim 9 is implemented.

Images