Sensor management method and apparatus, vehicle control system, and autonomous vehicle

Abstract

The present disclosure provides a sensor management method and device, a vehicle control system and an autonomous vehicle, and relates to the technical field of computers, in particular to the field of autonomous driving, operating systems and sensors. The specific implementation scheme is: receiving a sensor acquisition request, wherein the sensor acquisition request includes a sensor type to be acquired; and searching for a sensor object corresponding to the sensor type. In the embodiment of the present disclosure, the sensor object created by the plug-in corresponding to the sensor type can be searched according to the sensor type to be acquired in the sensor acquisition request, which is beneficial to more types of sensors.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the fields of autonomous driving, operating systems, and sensors. Background Art

[0002] In applications such as autonomous vehicles, a wide variety of sensor types and models are required. Directly integrating sensors requires writing adapters based on the sensor type, which increases development workload and reduces efficiency. Summary of the Invention

[0003] The present disclosure provides a sensor management method, device, vehicle control system, and autonomous driving vehicle.

[0004] According to one aspect of the present disclosure, a sensor management method is provided, including:

[0005] Receive a sensor acquisition request, where the sensor acquisition request includes the type of sensor to be acquired;

[0006] Find the sensor object corresponding to the sensor type. The sensor object is created using the sensor plug-in corresponding to the sensor type.

[0007] According to another aspect of the present disclosure, a sensor management device is provided, comprising:

[0008] A receiving module, configured to receive a sensor acquisition request, wherein the sensor acquisition request includes a sensor type to be acquired;

[0009] The first search module is used to search for a sensor object corresponding to the sensor type, where the sensor object is created using a sensor plug-in corresponding to the sensor type.

[0010] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0011] at least one processor; and

[0012] A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any embodiment of the present disclosure.

[0013] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method of any embodiment of the present disclosure.

[0014] According to another aspect of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method of any embodiment of the present disclosure is implemented.

[0015] According to another aspect of the present disclosure, a vehicle control system is provided, including: an electronic device including any embodiment of the present disclosure.

[0016] According to another aspect of the present disclosure, an autonomous driving vehicle is provided, comprising: an electronic device according to any embodiment of the present disclosure.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0019] Figure 1 is a flow chart of a sensor management method according to an embodiment of the present disclosure;

[0020] Figure 2 is a flow chart of a sensor management method according to another embodiment of the present disclosure;

[0021] Figure 3 is a flow chart of a sensor management method according to another embodiment of the present disclosure;

[0022] Figure 4 is a flow chart of a sensor management method according to another embodiment of the present disclosure;

[0023] Figure 5 is a structural diagram of a sensor management device according to an embodiment of the present disclosure;

[0024] Figure 6 is a structural diagram of a sensor management device according to another embodiment of the present disclosure;

[0025] Figure 7 is a schematic diagram of a framework of a sensor management method according to an embodiment of the present disclosure;

[0026] Figure 8 is a flowchart of a specific example of a sensor management method according to an embodiment of the present disclosure;

[0027] Figure 9 is a block diagram of an electronic device for implementing the sensor management method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0029] Figure 1 FIG. 1 is a flow chart of a sensor management method according to an embodiment of the present disclosure. The method may include:

[0030] S101, receiving a sensor acquisition request, where the sensor acquisition request includes a sensor type to be acquired;

[0031] S102: Searching for a sensor object corresponding to the sensor type, where the sensor object is created using a sensor plug-in corresponding to the sensor type.

[0032] For example, a device operating system may include an application layer and an abstraction layer. The application layer may include various application programs (APPs, also referred to as applications). The abstraction layer may include a sensor abstraction layer. The sensor abstraction layer may include an abstraction layer interface for interacting with the application layer. Applications in the application layer can access required sensor objects through this interface.

[0033] In an embodiment of the present disclosure, if a user opens a sensor-related application, the application can send a sensor acquisition request from the application layer to the abstract layer through the abstract layer interface. Sensor acquisition requests, such as sensor access requests and sensor object acquisition requests, can be used to obtain the sensor objects required by the application. After receiving the sensor acquisition request, the abstract layer interface can parse the request and obtain information such as the sensor type. Then, based on the sensor type, the sensor abstract layer searches to see if a sensor object corresponding to that sensor type is already stored. If an application has previously accessed a sensor object corresponding to that sensor type, the sensor abstract layer may already have an object generated by a sensor plug-in corresponding to that sensor type stored in it.

[0034] In the embodiments of the present disclosure, different application scenarios may include different sensor types. For example, cameras, radars, infrared, positioning, inertial navigation and other types of sensors may be installed on autonomous vehicles. Among them, cameras may also be called cameras, image acquisition devices, etc., and autonomous vehicles include various types of cameras. Radars may include ultrasonic radars, millimeter wave radars, lidars and other types of radars. Positioning sensors may include global positioning systems (GPS), etc. Inertial navigation sensors may include inertial measurement units (IMU), etc.

[0035] In embodiments of the present disclosure, a sensor plugin may include software adaptations related to sensor hardware. A particular type of sensor plugin may include implementation code for adapting to that type of sensor hardware. For example, a camera plugin may be used to adapt to camera-type sensors (referred to as camera sensors). Another example may be a radar plugin used to adapt to radar-type sensors (referred to as radar sensors).

[0036] For example, a camera application sends a camera-related sensor acquisition request to the sensor abstraction layer through the abstraction layer interface. The sensor acquisition request carries the sensor type, such as "camera." The sensor abstraction layer can then check whether a camera object corresponding to the camera type already exists.

[0037] For example, a radar application sends a radar-related sensor acquisition request to the sensor abstraction layer through the abstraction layer interface. The sensor acquisition request carries the radar type, such as "radar." The sensor abstraction layer can then check whether a radar object corresponding to the radar type already exists.

[0038] In the disclosed embodiments, the sensor type requested in the sensor acquisition request can be used to search for sensor objects created by the corresponding plug-in for that sensor type. This facilitates application to a wider range of sensor types and enables unified management and adaptation of sensor plug-ins and sensor objects corresponding to multiple sensor types. For example, the sensor management method of the disclosed embodiments facilitates unified management and adaptation of the various sensors required for autonomous vehicles, reducing development complexity.

[0039] Figure 2 FIG2 is a flow chart of a sensor management method according to another embodiment of the present disclosure. The sensor management method of this embodiment may include one or more features of the above method embodiments. In one embodiment, the sensor management method further includes:

[0040] S201 : When a sensor object corresponding to the sensor type is found, an output interface is called to return the sensor object.

[0041] In one example, the abstraction layer interface in the sensor abstraction layer may include an output interface. If a sensor object corresponding to a sensor type has already been generated and stored in the sensor abstraction layer, the output interface can be called to return the sensor object to an application in the application layer. The application layer application can then use the sensor object to collect the required data using the sensor hardware. For example, if a camera object corresponding to the camera type is found in the sensor abstraction layer, the output interface can be called to return the camera object to the camera application. For another example, if a radar object corresponding to the radar type is found in the sensor abstraction layer, the output interface can be called to return the radar object to the radar application. In the disclosed embodiments, after finding a sensor object created by a plug-in corresponding to the sensor type, the sensor object is returned via the output interface. This facilitates unified management and adaptation of sensor plug-ins and sensor objects based on sensor type, making it applicable to a wider range of sensor types. Furthermore, the sensor abstraction layer can isolate the dynamic library containing the sensor plug-in from the application layer, decoupling application development at the application layer from sensor plug-in development, facilitating parallel development.

[0042] In one embodiment, the sensor acquisition request further includes a sensor plug-in path. The method further includes: S202 , searching for a sensor plug-in according to the sensor plug-in path if no sensor object corresponding to the sensor type is found.

[0043] In one example, if the generated sensor object corresponding to the sensor type is not saved in the sensor abstraction layer, the sensor plug-in path in the sensor acquisition request may be used to search for a sensor plug-in corresponding to the sensor type in the sensor abstraction layer.

[0044] In the embodiment of the present disclosure, searching for a sensor plug-in according to the sensor plug-in path is helpful for quickly finding whether there is a loaded sensor plug-in, thereby speeding up the search.

[0045] In one embodiment, the method further includes: S203 , when a loaded sensor plug-in is found according to the sensor plug-in path, creating a sensor object using the loaded sensor plug-in.

[0046] In one example, if a sensor plug-in corresponding to the sensor type exists in the sensor abstraction layer, it means that the sensor plug-in corresponding to the sensor type has been loaded from the dynamic library, and the sensor object can be created using the sensor plug-in.

[0047] In the disclosed embodiments, a dynamic library can pre-store multiple types of sensor plug-ins. Sensors can be categorized, and each type of sensor plug-in may be compatible with multiple different models of sensor hardware. For example, a camera plug-in may be compatible with multiple models of camera hardware.

[0048] In the disclosed embodiment, after the loaded sensor plug-in is found according to the sensor plug-in path, it is beneficial to reuse the loaded sensor plug-in to create a sensor object, speed up the acquisition of the sensor object, and save system resources.

[0049] In one embodiment, the method further comprises:

[0050] S204: If no loaded sensor plug-in is found according to the sensor plug-in path, load the sensor plug-in corresponding to the sensor type from the dynamic library through the plug-in management interface;

[0051] S205: Create a sensor object using the loaded sensor plug-in.

[0052] In the disclosed embodiments, the plug-in management interface can be used to parse the sensor plug-in path and load the sensor plug-in corresponding to the sensor type from the dynamic library corresponding to the sensor plug-in path. This facilitates unified management of various sensor plug-in types in the dynamic library, enabling dynamic loading of sensor plug-ins and conserving system resources. Furthermore, by eliminating the need to restrict the specific implementation code used by sensor plug-ins, development complexity can be reduced, enabling parallel development and improving development efficiency. This can significantly reduce development workload for application scenarios where sensors may be rapidly iterated in parallel, such as autonomous driving scenarios.

[0053] In one embodiment, the sensor plug-in includes a sensor interface implementation code. In S203, creating a sensor object using the loaded sensor plug-in includes: calling a general sensor interface and / or a classified sensor interface to create a sensor object according to the sensor interface implementation code.

[0054] In the disclosed embodiments, the sensor interface implementation code may include code for various interfaces used to implement the specific functions of the sensor. Different types of sensors may include different interfaces, and different interfaces generally require different code to implement. In the disclosed embodiments, simply developing the implementation code for each interface of a particular sensor allows the generation of sensor objects containing these interfaces at the sensor abstraction layer, without having to focus on the implementation logic for the specific sensor functions. Therefore, by calling generic sensor interfaces and / or categorized sensor interfaces through the sensor interface implementation code, sensor objects can be created quickly and easily, facilitating unified categorized management of sensors and reducing development costs.

[0055] In one example, after a sensor plug-in is loaded into the sensor abstraction layer, the sensor interface implementation code in the sensor plug-in can be used to determine which general sensor interfaces and which classified sensor interfaces are required for this type of sensor. Then, using these general sensor interfaces and classified sensor interfaces, a sensor object is generated.

[0056] In one embodiment, the universal sensor interface may include functional interfaces that may be used by various types of sensors, such as one or more interfaces for starting acquisition, stopping acquisition, storage, input, and output.

[0057] In one embodiment, the categorized sensor interfaces include sensor interfaces required by autonomous vehicles. The categorized sensor interfaces required by autonomous vehicles may vary depending on the sensor type. For example, a camera sensor includes interfaces for recording and playback. Another example is a radar sensor that includes interfaces for ranging, speed, and angle measurement. Another example is a GPS that includes an interface for acquiring positioning information. By setting the categorized sensor interfaces required by autonomous vehicles, the sensor plug-ins for autonomous vehicles can be dynamically managed, reducing the difficulty of developing sensor plug-ins for autonomous vehicles.

[0058] In one embodiment, the sensor acquisition request further includes sensor parameters. The method further includes: S206 , initializing the sensor object created by the sensor plug-in using the sensor parameters.

[0059] In one embodiment, the method further includes: S207 , calling the output interface to return the initialized sensor object.

[0060] For example, sensor parameters may include initialization parameters. Different sensor types may have different initialization parameters. For example, the initialization parameters for a camera sensor include the camera type. In the sensor abstraction layer, when a sensor object is created using a sensor plugin, it must be initialized using the initialization parameters for that sensor type. For example, the camera object must be initialized based on parameters such as the camera model.

[0061] For example, sensor parameters can include not only initialization parameters but also other parameters such as communication methods. Therefore, in addition to initializing the camera, other necessary operations can also be performed. For example, for a TCP-based camera object, you can also create some TCP connections based on the sensor plug-in, communicate with the sensor hardware through the TCP connection, and return the sensor object to the application layer after successful communication.

[0062] In one example, the sensor abstraction layer uses a sensor plugin to create a new sensor object, initializes the sensor object, and then calls an output interface to return the initialized sensor object to the application layer. Applications in the application layer then use the sensor object to control the corresponding sensor hardware for various data collection tasks.

[0063] In the embodiment of the present disclosure, operations such as initialization are performed on a sensor object created using a sensor plug-in, so that the sensor object can be directly controlled by an application at the application layer to collect various data.

[0064] Figure 3 FIG2 is a flow chart of a sensor management method according to another embodiment of the present disclosure. The sensor management method of this embodiment may include one or more features of the above method embodiments. In one embodiment, the method further includes:

[0065] S301: Control the uninstallation of the sensor plug-in according to the usage of the sensor plug-in.

[0066] For example, in the sensor abstraction layer, usage flags and / or counters can be set for each sensor plugin to count how many applications are currently using the corresponding sensor plugin for each sensor type. If one or more sensor plugins meet the uninstall criteria, the plugin can be uninstalled. Dynamically controlling the uninstallation of sensor plugins helps reduce resource usage by unnecessary sensor plugins and conserve system resources.

[0067] In one embodiment, controlling uninstallation of the sensor plug-in according to usage of the sensor plug-in includes: uninstalling the sensor plug-in and / or deleting a sensor object corresponding to the sensor plug-in when detecting that no application uses the sensor plug-in.

[0068] In an embodiment of the present disclosure, if a usage flag or counter of a certain sensor plug-in indicates that the sensor plug-in and / or the sensor object created by the sensor plug-in is currently used by 0 applications, the sensor plug-in can be uninstalled and / or the sensor object corresponding to the sensor plug-in can be deleted.

[0069] In the embodiment of the present disclosure, by detecting the use of sensor plug-ins by applications, the uninstallation of unused sensor plug-ins can be dynamically controlled to ensure the normal operation of the application, and it is beneficial to reduce the resource occupation of unnecessary sensor plug-ins and save system resources.

[0070] In one embodiment, when it is detected that no application uses the sensor plug-in, the sensor plug-in is uninstalled and / or the sensor object corresponding to the sensor plug-in is deleted, including: when it is detected that no application uses the sensor plug-in within a set time range, the sensor plug-in is uninstalled and / or the sensor object corresponding to the sensor plug-in is deleted.

[0071] In the disclosed embodiments, sensor plug-in usage can be detected according to a set time range. For example, it can be detected whether a certain sensor plug-in has been used by an application within a certain time period, such as within 2 hours. In another example, it can be detected whether a certain sensor plug-in has been used by an application within a certain time period, such as 23:00 to 7:00.

[0072] If the usage flag or counter of a certain sensor plug-in indicates that the sensor plug-in and / or the sensor object created by the sensor plug-in is used by 0 applications within a set time range, the sensor plug-in can be uninstalled and / or the sensor object corresponding to the sensor plug-in can be deleted.

[0073] In the embodiment of the present disclosure, by detecting the use of sensor plug-ins by applications within a set time range, it is possible to dynamically control the uninstallation of unused sensor plug-ins in a timely manner, ensure the normal operation of the application, and help reduce the resource usage of unnecessary sensor plug-ins and save system resources.

[0074] Figure 4 This is a flow chart of a sensor management method according to another embodiment of the present disclosure. The sensor management method of this embodiment may include one or more features of the above-mentioned method embodiments. In one embodiment, the sensor plug-in also includes data parsing implementation code, and the method further includes:

[0075] S401 , according to the data parsing implementation code of the sensor plug-in, calling the data parsing interface to parse the data of each sensor object.

[0076] In one example, the data parsing interface can perform operations such as serialization and deserialization on data. The data parsing implementation code can provide methods for parsing sensor data. The data parsing implementation code can then call the data parsing interface to parse the sensor data. For example, each sensor needs to output raw data in a uniformly defined data format. This raw data can be sensor-independent. The data parsing interface can then parse this raw data to produce typed sensor data.

[0077] In the embodiment of the present disclosure, by using the data parsing implementation code included in the sensor plug-in and calling the data parsing interface, the data of each sensor object can be accurately parsed, which is conducive to unified management and adaptation of multiple types of sensors.

[0078] Figure 5 FIG. 1 is a schematic diagram of the structure of a sensor management device according to an embodiment of the present disclosure. The device may include:

[0079] The receiving module 501 is configured to receive a sensor acquisition request, where the sensor acquisition request includes a sensor type to be acquired;

[0080] The first search module 502 is configured to search for a sensor object corresponding to the sensor type, where the sensor object is created using a sensor plug-in corresponding to the sensor type.

[0081] In the disclosed embodiment, the first search module can search for sensor objects created by the corresponding plug-in based on the sensor type requested in the sensor acquisition request. This facilitates application to a wider range of sensor types and enables unified management and adaptation of sensor plug-ins and sensor objects corresponding to multiple sensor types. For example, the sensor management method of the disclosed embodiment can facilitate unified management and adaptation of the various sensors required for autonomous vehicles, reducing development complexity.

[0082] Figure 6 : is a structural diagram of a sensor management device according to another embodiment of the present disclosure. The sensor management device of this embodiment may include one or more features of the above-mentioned device embodiment. In one embodiment, the sensor management device also includes: a first output module 601, which is used to call the output interface to return the sensor object when a sensor object corresponding to the sensor type is found. In the embodiment of the present disclosure, after finding the sensor object created by the plug-in corresponding to the sensor type, the first output module returns the sensor object through the output interface, which is conducive to unified management and adaptation of sensor plug-ins and sensor objects based on sensor types, and is applicable to more types of sensors.

[0083] In one embodiment, the sensor acquisition request also includes a sensor plugin path, and the apparatus further includes a second search module 602 configured to search for a sensor plugin based on the sensor plugin path if no sensor object corresponding to the sensor type is found. In the disclosed embodiment, the second search module searches for a sensor plugin based on the sensor plugin path, which facilitates rapid retrieval of loaded sensor plugins and speeds up the search.

[0084] In one embodiment, the apparatus further includes a first creation module 603 configured to, upon locating a loaded sensor plug-in according to the sensor plug-in path, create a sensor object using the loaded sensor plug-in. In the disclosed embodiment, upon locating a loaded sensor plug-in according to the sensor plug-in path, the first creation module can reuse the loaded sensor plug-in to create the sensor object, thereby accelerating the acquisition of the sensor object and conserving system resources.

[0085] In one embodiment, the device further comprises:

[0086] The loading module 604 is configured to load the sensor plug-in corresponding to the sensor type from the dynamic library through the plug-in management interface if no loaded sensor plug-in is found according to the sensor plug-in path;

[0087] The second creation module 605 is configured to create a sensor object using the loaded sensor plug-in.

[0088] In the disclosed embodiment, the loading module can parse the sensor plugin path through the plugin management interface and load the sensor plugin corresponding to the sensor type from the dynamic library corresponding to the sensor plugin path. This facilitates unified management of various sensor plugin types in the dynamic library, enabling dynamic loading of sensor plugins and conserving system resources.

[0089] In one embodiment, the sensor plug-in includes sensor interface implementation code, and creating a sensor object includes: invoking a universal sensor interface and / or a classified sensor interface to create the sensor object based on the sensor interface implementation code. For example, the first creation module 603 is specifically configured to invoke a universal sensor interface and / or a classified sensor interface to create the sensor object based on the sensor interface implementation code. In another example, the second creation module 605 is specifically configured to invoke a universal sensor interface and / or a classified sensor interface to create the sensor object based on the sensor interface implementation code. Invoking a universal sensor interface and / or a classified sensor interface through the sensor interface implementation code allows for convenient and efficient creation of sensor objects, facilitating unified classification management of sensors and reducing development costs.

[0090] In one embodiment, the classified sensor interface includes the sensor interface required by the autonomous vehicle. By setting the classified sensor interface required by the autonomous vehicle, the sensor plug-in of the autonomous vehicle can be dynamically managed, reducing the difficulty of developing the sensor plug-in for the autonomous vehicle.

[0091] In one embodiment, the sensor acquisition request further includes sensor parameters, and the apparatus further includes: an initialization module 606 configured to initialize the sensor object created by the sensor plug-in using the sensor parameters.

[0092] In one embodiment, the device further comprises:

[0093] The second output module 607 is used to call the output interface to return the initialized sensor object.

[0094] In the embodiment of the present disclosure, the initialization module performs operations such as initialization on the sensor object created by the sensor plug-in, so that the sensor object can be directly controlled by the application of the application layer to perform various data collection.

[0095] In one embodiment, the device further includes an uninstallation module 608 configured to control the uninstallation of the sensor plug-in based on the usage of the sensor plug-in. The uninstallation module can dynamically control the uninstallation of the sensor plug-in, thereby reducing the resource usage of unnecessary sensor plug-ins and saving system resources.

[0096] In one embodiment, the uninstall module is specifically configured to uninstall the sensor plugin and / or delete the sensor object corresponding to the sensor plugin upon detecting that no application is using the sensor plugin. In the disclosed embodiment, the uninstall module, by detecting whether an application is using the sensor plugin, can dynamically control the uninstallation of unused sensor plugins, ensuring the normal operation of the application, reducing resource usage by unnecessary sensor plugins, and conserving system resources.

[0097] In one embodiment, the uninstall module is specifically configured to uninstall the sensor plugin and / or delete the sensor object corresponding to the sensor plugin if it detects that no application has used the sensor plugin within a set time range. In the disclosed embodiment, by detecting whether an application has used the sensor plugin within the set time range, the uninstall module can dynamically control the uninstallation of unused sensor plugins in a timely manner, ensuring the normal operation of the application, reducing unnecessary sensor plugin resource usage, and conserving system resources.

[0098] In one embodiment, the sensor plug-in also includes data parsing implementation code, and the device further includes a data parsing module 609 configured to, based on the sensor plug-in's data parsing implementation code, call a data parsing interface to parse data from each sensor object. In this disclosed embodiment, the data parsing module, through the data parsing implementation code included in the sensor plug-in and calling the data parsing interface, can accurately parse the data from each sensor object, facilitating unified management and adaptation of multiple sensor types.

[0099] For the description of specific functions and examples of each module of the sensor management device disclosed herein, reference may be made to the description of corresponding steps in the above-mentioned sensor management method embodiment, which will not be repeated here.

[0100] In one application scenario, each sensor type is abstracted independently, with its own interface and data interface defined for each. The application layer is developed based on these interfaces, integrating sensor implementations into a dynamic library. For example, for the same type of GPS sensor, a unified access interface is defined. Different projects implement this abstracted GPS interface based on specific GPS models and export specific GPS dynamic libraries. The application layer accesses the GPS using this unified interface.

[0101] The disclosed embodiments not only abstract sensors but also provide some general abstractions for all types of sensors, enabling the development of general sensor tools at the application level. Furthermore, the disclosed embodiments can load dynamic libraries based on application requirements, enabling functions such as dynamic loading and switching of sensors.

[0102] The present disclosure provides a sensor abstraction layer framework, for example, a vehicle sensor abstraction layer framework, which can use the sensor management method of the above-mentioned embodiment of the present disclosure.

[0103] like Figure 7 As shown, the sensor abstraction layer structure can include the following parts:

[0104] Abstraction layer interface 701: The sensor abstraction layer can interact with the application layer through the abstraction layer interface, such as creating and releasing sensor interfaces. The abstraction layer interface can also be called an output interface.

[0105] Plug-in management interface 702: This interface can provide an interface for the sensor abstraction layer to uniformly manage sensor plug-ins, and this interface can be implemented by the sensor plug-in.

[0106] Universal sensor interface 703: also known as sensor universal interface, provides a universal sensor access interface for sensors. All sensors that need to be integrated into the sensor abstraction layer need to implement this universal interface.

[0107] Classification sensor interface 704: also known as the sensor classification interface, defines a unified abstract interface for commonly used sensors in scenarios such as autonomous driving. Different types of sensor plug-ins need to implement interfaces consistent with their own types.

[0108] Data parsing interface 705: provides a sensor data parsing interface. The sensor plug-in implements this interface to parse the raw sensor data output by the plug-in.

[0109] The embodiment of the present disclosure provides a method for dynamically managing different sensor plug-ins. The specific process is as follows: Figure 8 shown.

[0110] S801. Obtain sensor: The application layer may access the sensor through the abstraction layer interface, for example, by sending a sensor acquisition request to the sensor abstraction layer.

[0111] S802: Determine whether a sensor object has been created. For example, if the sensor acquisition request includes a sensor type, the sensor abstraction layer can be checked to see if a sensor object corresponding to this sensor type has been stored. If so, it indicates that a sensor object corresponding to this sensor type has been created, and S806 can be executed. If not, it indicates that a sensor object corresponding to this sensor type has not been created, and S803 can be executed.

[0112] S803: Determine whether the sensor plug-in has been loaded. For example, the sensor acquisition request includes a plug-in path. Based on the plug-in path, the sensor abstraction layer can be checked to see whether a sensor plug-in corresponding to this sensor type is stored. If so, it indicates that the sensor plug-in corresponding to this sensor type has been loaded, and S805 can be executed. If not, it indicates that the sensor plug-in corresponding to this sensor type has not been loaded, and S804 can be executed.

[0113] S804: Load the sensor plug-in. For example, load the sensor plug-in corresponding to the sensor type in the dynamic library according to the plug-in path.

[0114] S805. Use the sensor plug-in to obtain a sensor object. For example, the sensor interface implementation code in the sensor plug-in calls the sensor general interface and / or sensor classification interface to construct the sensor object. Sensor parameters (via a sensor acquisition request or other information passed from the application layer to the sensor abstraction layer) are passed to the sensor plug-in, and the sensor object is initialized.

[0115] S806: Return the sensor object. For example, the sensor abstraction layer may return the sensor object via an abstraction layer interface, such as an output interface.

[0116] The advantages of this example framework include at least one of the following:

[0117] Unifying sensor interfaces simplifies application code development. A unified sensor interface and usage process simplifies application-layer code logic. This decouples the application layer from sensor implementation, reducing adaptation efforts associated with sensor changes.

[0118] Dynamically managing sensor plugins can avoid resource waste. Furthermore, plugin management further decouples application-layer and sensor implementation-layer development, improving parallel development efficiency. For example, autonomous vehicles utilize a wide variety of sensors. If dynamic libraries were statically loaded to load all the sensors they might use into memory, this could result in resource waste. Dynamically loading and unloading sensor plugins can avoid this resource hog.

[0119] By defining a common interface for sensors, it is easier to develop tools that use common sensor processing, such as serialized recording and playback of sensor data.

[0120] In one application scenario, autonomous driving control requires the use of many different sensors, including cameras, millimeter-wave radars, ultrasonic radars, GPS, IMUs, etc. Since algorithms and related sensors may iterate in parallel and very quickly, a lot of development work is required to adapt to changes in algorithms and sensors. The efficient sensor abstraction framework provided by the disclosed embodiment can quickly adapt to new sensors by unifying the calling interface and usage process of sensors, support parallel development, and improve development efficiency. In addition, through dynamic management of plug-ins, resource waste is avoided.

[0121] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0122] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0123] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0124] like Figure 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the device 900 can also be stored in the RAM 903. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0125] Various components in the device 900 are connected to the I / O interface 905, including an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0126] The computing unit 901 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as a sensor management method. For example, in some embodiments, a sensor management method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the sensor management method described above can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform a sensor management method by any other appropriate means (e.g., by means of firmware).

[0127] According to another aspect of the present disclosure, a vehicle control system is provided, comprising: an electronic device according to any embodiment of the present disclosure. The vehicle control system may also be referred to as a vehicle system, a vehicle operating system, etc.

[0128] According to another aspect of the present disclosure, a vehicle is provided, comprising: an electronic device according to any embodiment of the present disclosure. For example, the vehicle may be an autonomous vehicle. The autonomous vehicle may include autonomous vehicles of various autonomous driving levels.

[0129] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0131] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0133] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0134] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0135] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0136] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A sensor management method, applied to an operating system of a device, wherein the operating system includes an application layer and an abstraction layer; the application layer includes various applications; the abstraction layer includes a sensor abstraction layer; the sensor abstraction layer includes an abstraction layer interface for interacting with the application layer; and applications in the application layer can access required sensor objects through the abstraction layer interface, comprising: The sensor abstraction layer receives the sensor acquisition request sent by the application layer through the abstraction layer interface, where the sensor acquisition request includes the sensor type to be acquired; The sensor abstraction layer searches whether a sensor object corresponding to the sensor type is stored in the sensor abstraction layer, where the sensor object is created using a sensor plug-in corresponding to the sensor type; If no sensor object corresponding to the sensor type is found in the sensor abstraction layer, searching, according to the sensor plug-in path in the sensor acquisition request, whether a sensor plug-in corresponding to the sensor type exists in the sensor abstraction layer; When a loaded sensor plug-in is found in the sensor abstraction layer according to the sensor plug-in path, a sensor object is created using the loaded sensor plug-in.

2. The method according to claim 1, further comprising: When a sensor object corresponding to the sensor type is found, the output interface is called to return the sensor object.

3. The method according to claim 1, further comprising: If no loaded sensor plug-in is found according to the sensor plug-in path, loading the sensor plug-in corresponding to the sensor type from the dynamic library through the plug-in management interface; Create a sensor object using the loaded sensor plugin.

4. The method according to claim 3, wherein: The sensor plug-in includes sensor interface implementation code, and creating a sensor object includes: According to the sensor interface implementation code, a general sensor interface and / or a classified sensor interface is called to create a sensor object.

5. The method according to claim 4, wherein The classified sensor interfaces include sensor interfaces required for autonomous driving vehicles.

6. The method according to any one of claims 1 to 5, wherein The sensor acquisition request also includes sensor parameters, and the method further includes: Initialize the sensor object created by the sensor plug-in using the sensor parameters; Calling the output interface returns the initialized sensor object.

7. The method according to any one of claims 1 to 5, further comprising: The uninstallation of the sensor plug-in is controlled according to the usage of the sensor plug-in.

8. The method according to claim 7, wherein: Controlling uninstallation of the sensor plug-in according to usage of the sensor plug-in includes: When it is detected that no application uses the sensor plug-in, the sensor plug-in is uninstalled and / or the sensor object corresponding to the sensor plug-in is deleted.

9. The method according to claim 8, wherein When it is detected that no application uses the sensor plug-in, uninstalling the sensor plug-in and / or deleting the sensor object corresponding to the sensor plug-in includes: When it is detected that no application uses the sensor plug-in within a set time range, the sensor plug-in is uninstalled and / or the sensor object corresponding to the sensor plug-in is deleted.

10. The method according to any one of claims 1 to 5, wherein The sensor plug-in also includes data parsing implementation code, and the method further includes: According to the data parsing implementation code of the sensor plug-in, the data parsing interface is called to parse the data of each sensor object.

11. A sensor management device, applied to an operating system of a device, wherein the operating system includes an application layer and an abstraction layer; the application layer includes various applications; the abstraction layer includes a sensor abstraction layer; the sensor abstraction layer includes an abstraction layer interface for interacting with the application layer; applications in the application layer can access required sensor objects through the abstraction layer interface, the device comprising: A receiving module, configured to receive a sensor acquisition request sent by the application layer through the abstract layer interface, wherein the sensor acquisition request includes a sensor type to be acquired; A first search module is configured to search whether a sensor object corresponding to the sensor type is stored in the sensor abstraction layer, where the sensor object is created using a sensor plug-in corresponding to the sensor type; a second search module configured to search, if no sensor object corresponding to the sensor type is found in the sensor abstraction layer, for a sensor plug-in corresponding to the sensor type in the sensor abstraction layer according to the sensor plug-in path in the sensor acquisition request; The first creation module is configured to create a sensor object using the loaded sensor plug-in when a loaded sensor plug-in is found in the sensor abstraction layer according to the sensor plug-in path.

12. The apparatus according to claim 11, further comprising: The first output module is configured to, when a sensor object corresponding to the sensor type is found, call an output interface to return the sensor object.

13. The apparatus according to claim 11, further comprising: A loading module, configured to load the sensor plug-in corresponding to the sensor type from a dynamic library through a plug-in management interface if no loaded sensor plug-in is found according to the sensor plug-in path; The second creation module is configured to create a sensor object using the loaded sensor plug-in.

14. The device according to claim 13, wherein The sensor plug-in includes sensor interface implementation code, and creating a sensor object includes: According to the sensor interface implementation code, a general sensor interface and / or a classified sensor interface is called to create a sensor object.

15. The device according to claim 14, wherein The classified sensor interfaces include sensor interfaces required for autonomous driving vehicles.

16. The device according to any one of claims 11 to 15, wherein The sensor acquisition request also includes sensor parameters, and the device further includes: An initialization module, configured to initialize the sensor object created by the sensor plug-in using the sensor parameters; The second output module is used to call the output interface to return the initialized sensor object.

17. The apparatus according to any one of claims 11 to 15, further comprising: The uninstallation module is used to control the uninstallation of the sensor plug-in according to the usage of the sensor plug-in.

18. The device according to claim 17, wherein The uninstallation module is specifically configured to uninstall the sensor plug-in and / or delete the sensor object corresponding to the sensor plug-in when it is detected that no application uses the sensor plug-in.

19. The device according to claim 18, wherein The uninstallation module is specifically configured to uninstall the sensor plug-in and / or delete the sensor object corresponding to the sensor plug-in when it is detected that no application uses the sensor plug-in within a set time range.

20. The device according to any one of claims 11 to 15, wherein The sensor plug-in also includes data parsing implementation code, and the device also includes: The data parsing module is used to implement the data parsing code according to the sensor plug-in and call the data parsing interface to parse the data of each sensor object.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

23. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

24. A vehicle control system comprising: Comprising the electronic device as claimed in claim 21.

25. An autonomous driving vehicle comprising: Comprising the electronic device as claimed in claim 21.

Images