A Sensor Control Method Based on Context Classification and an IoT Middleware

Abstract

An embodiment of the present invention provides a sensor control method based on context classification and an Internet of Things middleware, which relates to the technical field of the Internet of Things. The method includes: obtaining first type of information collected in real time by a first type of sensor and second type of information collected by a second type of sensor according to a current sampling frequency; wherein, there are differences in energy consumption during the information collection process between the first type of sensor and the second type of sensor; determining a current context category based on the first type of information and the second type of information; looking up, from a preset correspondence between context categories and sampling frequencies, a sampling frequency corresponding to the current context category as a target sampling frequency; controlling the second type of sensor to collect information according to the target sampling frequency; Compared with the prior art, by applying the solution provided by the embodiment of the present invention, it is possible to reduce the energy consumption of each sensor in the Internet of Things, so as to extend the service life of each sensor, thereby reducing the maintenance requirements for each sensor and lowering the maintenance cost.

Description

Technical Field

[0001] The present invention relates to the technical field of the Internet of Things, and particularly to a sensor control method based on context classification and an Internet of Things middleware. Background Art

[0002] With the continuous development of science and technology, the Internet of Things (IOT), an emerging technology, has developed rapidly. The so-called Internet of Things is the "Internet of everything connected", which is an extension and expansion of the Internet. It is a huge network formed by combining various information sensing devices with the network, realizing the interconnection and interoperability of people, machines, and things at any time and any place.

[0003] That is to say, the Internet of Things extends the concept of the Internet to the connection between physical devices and daily objects, thereby realizing the connection between physical devices and the real environment, which can usually be achieved through the Internet of Things middleware.

[0004] Among them, the so-called Internet of Things middleware is a type of software between the application system and the operating system. The Internet of Things middleware uses the basic services provided by the operating system to connect the application software of each part of the application system, thereby achieving the purpose of resource sharing and function sharing.

[0005] In addition, context awareness technology is an important component of the Internet of Things technology. The so-called context awareness technology means: based on the data information sensed by each sensor set at the bottom layer of the Internet of Things in the real environment, and then, after processing and judgment, informing the physical device of the context corresponding to the current real environment. Simply put, it enables physical devices to "perceive" the current context through sensors and related technologies.

[0006] In the actual application of the Internet of Things, considering that due to some relatively harsh and complex real environments, the maintenance of each sensor is difficult and costly. Therefore, how to reduce the energy consumption of each sensor in the Internet of Things to extend the service life of each sensor, thereby reducing the maintenance requirements of each sensor and lowering the maintenance cost has become an urgent problem to be solved. Summary of the Invention

[0007] The purpose of the embodiments of the present invention is to provide a sensor control method based on context classification and an Internet of Things middleware, which can reduce the energy consumption of each sensor in the Internet of Things to extend the service life of each sensor, thereby reducing the maintenance requirements of each sensor and lowering the maintenance cost. The specific technical solutions are as follows:

[0008] In a first aspect, the embodiments of the present invention provide a sensor control method based on context classification, which is applied to the Internet of Things middleware. The method includes:

[0009] Obtain the first type of information collected in real time by the first type of sensor, and the second type of information collected by the second type of sensor at the current sampling frequency;

[0010] Based on the first type of information and the second type of information, determine the current situation category; wherein, there are differences in the energy consumption of the first type of sensor and the second type of sensor during the information collection process;

[0011] Search in the preset correspondence between the situation category and the sampling frequency for the sampling frequency corresponding to the current situation category as the target sampling frequency;

[0012] Control the second type of sensor to collect information at the target sampling frequency.

[0013] Optionally, in a specific implementation manner, the method further includes:

[0014] Obtain a specified sampling frequency according to the target sampling frequency; wherein, the specified sampling frequency is related to the current situation category and has a multiple relationship with the target sampling frequency

[0015] Control the third type of sensor to determine the third type of information at the specified sampling frequency; wherein, there are differences in the energy consumption of the first type of sensor, the second type of sensor, and the third type of sensor during the information collection process.

[0016] Optionally, in a specific implementation manner, before the step of determining the current situation category based on the first type of information and the second type of information, the method includes:

[0017] Perform data preprocessing on the first type of information and / or the second type of information to obtain the preprocessed first type of information and / or second type of information;

[0018] The step of determining the current situation category based on the first type of information and the second type of information includes:

[0019] In the case of performing data preprocessing on the first type of information or the second type of information, determine the current situation category based on the preprocessed first type of information and the second type of information, or determine the current situation category based on the first type of information and the preprocessed second type of information;

[0020] In the case of performing data preprocessing on the first type of information and the second type of information, determine the current situation category based on the preprocessed first type of information and the preprocessed second type of information.

[0021] Optionally, in a specific implementation manner, before the step of controlling the second type of sensor to collect information at the target sampling frequency, the method further includes:

[0022] Determine whether the current situation category matches a preset topic subscription table; wherein, the topic subscription table is set based on the subscription and publishing services provided by the Internet of Things middleware;

[0023] If there is a match, control the second type of sensor to collect information according to the target sampling frequency.

[0024] Optionally, in a specific implementation manner, the determining the current situation category based on the first type of information and the second type of information includes:

[0025] Determine a target decision condition according to the first type of information and the second type of information; wherein, the target decision condition is used to divide the current real environment into corresponding situation states;

[0026] Search in a preset correspondence between decision conditions and situation categories for the situation category corresponding to the target decision condition as the current situation category.

[0027] Optionally, in a specific implementation manner, the determining the target decision condition according to the first type of information and the second type of information includes:

[0028] Determine respective first contribution degrees of each piece of the first type of information to forming the situation state corresponding to the current real environment, and respective second contribution degrees of each piece of the second type of information to forming the situation state;

[0029] Based on the determined respective first contribution degrees and respective second contribution degrees, determine a target decision condition for characterizing the situation state.

[0030] Optionally, in a specific implementation manner, the specified sampling frequency is: the target sampling frequency; the controlling the third type of sensor to determine the third type of information according to the specified sampling frequency includes:

[0031] Control the third type of sensor to collect the third type of information according to the target sampling frequency; or,

[0032] Control the third type of sensor to infer the third type of information according to the target sampling frequency based on the information currently collected by the first type of sensor, the information currently collected by the second type of sensor, and the historical information determined by the third type of sensor.

[0033] Optionally, in a specific implementation manner, the method further includes:

[0034] Search in a preset correspondence between situation categories and node modes for the node mode corresponding to the current situation category as the target node mode;

[0035] Control the second type of sensor to switch from the current node mode to the target node mode.

[0036] In a second aspect, an embodiment of the present invention provides an Internet of Things middleware, which includes: a context awareness module, an open service interface module, and a comprehensive application platform;

[0037] The context awareness module is used for:

[0038] Obtain the first type of information collected in real time by the first type of sensor and the second type of information collected by the second type of sensor according to the current sampling frequency; and determine the current context category based on the first type of information and the second type of information; wherein, there are differences in energy consumption during the information collection process between the first type of sensor and the second type of sensor;

[0039] Search for the sampling frequency corresponding to the current context category from the preset correspondence between the context category and the sampling frequency as the target sampling frequency; and send the target sampling frequency to the comprehensive application platform through the open service interface module;

[0040] The comprehensive application platform is used for:

[0041] Control the second type of sensor to collect information according to the target sampling frequency.

[0042] Optionally, in a specific implementation manner, the comprehensive application platform is further used for:

[0043] Obtain a specified sampling frequency according to the target sampling frequency; wherein, the specified sampling frequency is related to the current context category and has a multiple relationship with the target sampling frequency;

[0044] Control the third type of sensor to determine the third type of information according to the specified sampling frequency; wherein, there are differences in energy consumption during the information collection process between the first type of sensor, the second type of sensor, and the third type of sensor.

[0045] Optionally, in a specific implementation manner, the context awareness module is further used for:

[0046] Perform data preprocessing on the first type of information and / or the second type of information to obtain the preprocessed first type of information and / or second type of information;

[0047] The determining the current context category based on the first type of information and the second type of information includes:

[0048] In the case of performing data preprocessing on the first type of information or the second type of information, based on the preprocessed first type of information and the second type of information, determine the current situation category, or, based on the first type of information and the preprocessed second type of information, determine the current situation category;

[0049] In the case of performing data preprocessing on the first type of information and the second type of information, based on the preprocessed first type of information and the preprocessed second type of information, determine the current situation category.

[0050] Optionally, in a specific implementation manner, before sending the target sampling frequency to the integrated application platform through the open service interface module, the situation awareness module is further configured to:

[0051] Determine whether the current situation category matches a preset topic subscription table; wherein, the preset topic subscription table is set based on the subscription and publishing services provided by the situation awareness module;

[0052] If it matches, send the target sampling frequency to the integrated application platform through the open service interface module.

[0053] Optionally, in a specific implementation manner, the situation awareness module is further configured to:

[0054] Search for the node pattern corresponding to the current situation category from the preset correspondence between the situation category and the node pattern as the target node pattern; and send the target node pattern to the integrated application platform through the open service interface module;

[0055] The integrated application platform is further configured to control the second type of sensor to switch from the current node pattern to the target node pattern.

[0056] Optionally, in a specific implementation manner, the situation awareness module includes: a data preprocessing unit and a decision maker;

[0057] The data preprocessing unit is configured to perform data preprocessing on the first type of information and / or the second type of information to obtain the preprocessed first type of information and / or the second type of information;

[0058] The decider is configured to, when performing data preprocessing on the first type of information or the second type of information, determine the current context category based on the preprocessed first type of information and the second type of information, or determine the current context category based on the first type of information and the preprocessed second type of information; when performing data preprocessing on the first type of information and the second type of information, determine the current context category based on the preprocessed first type of information and the preprocessed second type of information; and look up, from a preset correspondence between context categories and sampling frequencies, the sampling frequency corresponding to the current context category as the target sampling frequency.

[0059] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of any one of the sensor control methods based on context classification provided in the first aspect are implemented.

[0060] In a fourth aspect, an embodiment of the present invention provides a computer program product containing instructions, which, when running on a computer, causes the computer to execute the steps of any one of the sensor control methods based on context classification provided in the first aspect.

[0061] Advantages of the embodiments of the present invention:

[0062] By applying the solution provided in the embodiments of the present invention, in the actual application of the Internet of Things, various sensors required in the application scenario can be designed according to the application scenario where the Internet of Things is located. Furthermore, based on the types of information collected by each sensor and the energy consumption differences of each sensor during the information collection process, each sensor can be divided into a first type of sensor, a second type of sensor, and a third type of sensor. Among them, the first type of sensor performs information collection in real time, the second type of sensor performs information collection at a sampling frequency, and the third type of sensor performs information determination at another sampling frequency. Among them, the sampling frequency adopted by the second type of sensor and the sampling frequency adopted by the third type of sensor may be the same or different, and the energy consumption of the first type of sensor, the second type of sensor, and the third type of sensor during the information collection process is different.

[0063] In this way, the Internet of Things middleware can obtain the first type of information collected in real time by the first type of sensor and the second type of information collected by the second type of sensor at the current sampling frequency. Furthermore, based on the obtained first type of information and the second type of information, the current context where the Internet of Things is located can be determined, and thus the current context category can be obtained. Then, the sampling frequency corresponding to the above current context category can be looked up from the preset correspondence between context categories and sampling frequencies as the target sampling frequency. After that, the second type of sensor can be controlled to perform information collection at the target sampling frequency.

[0064] Based on this, when applying the solution provided by the embodiments of the present invention in the actual application of the Internet of Things, different sampling frequencies can be allocated to different sensors according to the requirements of the application scenario and the energy consumption of each sensor. Moreover, the sampling frequencies of each sensor can be adjusted according to the changes in the current situation, so that the sampling frequencies of each sensor can match the current situation category. In this way, through the combination of context awareness technology and the Internet of Things middleware, different sensors can be in different working modes, achieving the effect of reducing the energy consumption of each sensor in the Internet of Things, extending the service life of each sensor, reducing the maintenance requirements of each sensor, and lowering the maintenance cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other embodiments based on these drawings.

[0066] Figure 1 A flowchart of a sensor control method based on context classification provided by an embodiment of the present invention;

[0067] Figure 2 For Figure 1 A flowchart of a specific implementation manner of S102 in

[0068] Figure 3 Another flowchart of a sensor control method based on context classification provided by an embodiment of the present invention;

[0069] Figure 4 A structural diagram of an Internet of Things middleware provided by an embodiment of the present invention;

[0070] Figure 5 An architecture diagram of an Internet of Things middleware provided by an embodiment of the present invention in a vehicle collision avoidance scenario based on the Internet of Things. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0071] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art based on this application belong to the scope of protection of the present invention.

[0072] In the practical applications of the Internet of Things (IoT), considering that due to some relatively harsh and complex real-world environments, the maintenance of each sensor is difficult and costly. Therefore, how to reduce the energy consumption of each sensor in the IoT to extend the service life of each sensor, thereby reducing the maintenance requirements and costs of each sensor, has become an urgent problem to be solved currently.

[0073] To solve the above problems, an embodiment of the present invention provides a sensor control method based on context classification.

[0074] Among them, this method is applied to the IoT middleware, and moreover, this method can be applicable to any IoT application scenario that needs to connect the application system and the operating system with the help of the IoT middleware. For example, vehicle collision avoidance scenarios based on the IoT, forest fire warning scenarios based on the IoT, smart home control scenarios based on the IoT, coal mine safety monitoring scenarios based on the IoT, etc.

[0075] In the embodiment of the present invention, in various application scenarios, each sensor required for each type of application scenario can be designed. Furthermore, according to the information types collected by each sensor and the energy consumption of each sensor during the information collection process, the sensors required for this application scenario are classified. Thus, the sensors are divided into the first type of sensors, the second type of sensors, and the third type of sensors. And there are differences in the energy consumption of the first type of sensors, the second type of sensors, and the third type of sensors during the information collection process.

[0076] Among them, the first type of sensors is also called basic sensors, which can support the basic functions of the application scenario. When the IoT in the application scenario starts, this type of sensor must always work, that is, collect information in real time. Usually, optionally, the first type of sensors are sensors that are only related to the hardware or relevant processes in the application scenario. For example, the time sensor in the vehicle collision avoidance scenario based on the IoT.

[0077] The second type of sensors is also called low-to-medium energy consumption sensors. The information they collect can remain unchanged for a period of time. Therefore, the sampling frequency of this type of sensor can be adjusted according to the current context. For example, the meteorological sensor, direction sensor, and speed sensor in the vehicle collision avoidance scenario based on the IoT.

[0078] The third type of sensors is also called high energy consumption sensors. The information they collect can change continuously. However, in the application scenario, it is not necessary to keep this type of sensor running all the time. That is to say, this type of sensor can be run once every certain period of time. For example, the GPS (Global Positioning System) sensor in the vehicle collision avoidance scenario based on the IoT.

[0079] A sensor control method based on context classification provided by an embodiment of the present invention may include the following steps:

[0080] Obtain the first type of information collected in real time by the first type of sensors and the second type of information collected by the second type of sensors at the current sampling frequency; wherein, there are differences in the energy consumption of the first type of sensors and the second type of sensors during the information collection process;

[0081] Based on the first type of information and the second type of information, determine the current context category;

[0082] Search in the preset correspondence between context categories and sampling frequencies for the sampling frequency corresponding to the current context category as the target sampling frequency;

[0083] Control the second type of sensors to collect information at the target sampling frequency.

[0084] As can be seen above, by applying the solution provided by the embodiment of the present invention, in the actual application of the Internet of Things, various sensors required in the application scenario can be designed according to the application scenario where the Internet of Things is located. Furthermore, based on the types of information collected by each sensor and the differences in the energy consumption of each sensor during the information collection process, each sensor can be divided into the first type of sensors, the second type of sensors, and the third type of sensors. Among them, the first type of sensors collect information in real time, the second type of sensors collect information at a sampling frequency, and the third type of sensors determine information at another sampling frequency. Among them, the sampling frequency adopted by the second type of sensors and the sampling frequency adopted by the third type of sensors may be the same or different, and there are differences in the energy consumption of the first type of sensors, the second type of sensors, and the third type of sensors during the information collection process.

[0085] In this way, the Internet of Things middleware can obtain the first type of information collected in real time by the first type of sensors and the second type of information collected by the second type of sensors at the current sampling frequency. Furthermore, based on the obtained first type of information and the second type of information, the context where the Internet of Things is currently located can be determined, and thus the current context category can be obtained. Then, the sampling frequency corresponding to the above current context category can be searched in the preset correspondence between context categories and sampling frequencies as the target sampling frequency. After that, the second type of sensors can be controlled to collect information at the target sampling frequency.

[0086] Based on this, when applying the solution provided by the embodiments of the present invention in the actual application of the Internet of Things, different sampling frequencies can be allocated to different sensors according to the requirements of the application scenario and the energy consumption of each sensor. Moreover, the sampling frequencies of each sensor can be adjusted according to the changes in the current situation, so that the sampling frequencies of each sensor can match the current situation category. In this way, through the combination of context awareness technology and the Internet of Things middleware, different sensors can be in different working modes, achieving the effect of reducing the energy consumption of each sensor in the Internet of Things, extending the service life of each sensor, reducing the maintenance requirements of each sensor, and lowering the maintenance cost.

[0087] In addition, in the embodiments of the present invention, each sensor can be imported into the Internet of Things in advance, and the combinations of the first type of sensors, the second type of sensors, and the third type of sensors corresponding to different application scenarios can be set in advance. Thus, in the actual application, the current application scenario can be determined according to the detected location information. Furthermore, the combinations of the first type of sensors, the second type of sensors, and the third type of sensors corresponding to this application scenario can be matched among the pre-imported sensors. Thus, the Internet of Things middleware can implement a sensor control method based on context classification provided by the embodiments of the present invention based on the matched combinations of the first type of sensors, the second type of sensors, and the third type of sensors.

[0088] Based on this, a sensor control method based on context classification provided by the embodiments of the present invention can be applied to a wider range of application scenarios without being restricted by the application scenario.

[0089] Next, with reference to the accompanying drawings, a sensor control method based on context classification provided by the embodiments of the present invention will be specifically described.

[0090] Figure 1 FIG. is a schematic flowchart of a sensor control method based on context classification provided by the embodiments of the present invention. As Figure 1 shown, the method may include the following steps:

[0091] S101: Obtain the first type of information collected in real time by the first type of sensors, and the second type of information collected by the second type of sensors at the current sampling frequency;

[0092] After determining the first type of sensors, the second type of sensors, and the third type of sensors involved in the application scenario, the first type of sensors can collect the first type of information in the application scenario in real time, while the second type of sensors can collect the second type of information in the application scenario at the current sampling frequency. Thus, the first type of sensors can send the collected second type of information to the Internet of Things middleware.

[0093] In this way, the Internet of Things middleware can obtain the first type of information collected in real time by the first type of sensors and the second type of information collected by the second type of sensors at the current sampling frequency. Among them, there are differences in the energy consumption of the first type of sensors and the second type of sensors during the information collection process.

[0094] S102: Determine the current situation category based on the first type of information and the second type of information;

[0095] After obtaining the above first type of information and the second type of information, the Internet of Things middleware can determine the current situation based on the above first type of information and the second type of information, and thus determine the current situation category.

[0096] For example, in the vehicle collision avoidance scenario based on the Internet of Things, the first type of sensor is a time sensor, and the second type of sensors include a meteorological sensor, a direction sensor, and a speed sensor.

[0097] Exemplarily, when the first type of information obtained is: morning, and the second type of information includes: sunny day, straight road, and 20 km / s, it can be determined that the current situation is: a situation where a traffic accident is less likely to occur, and thus, the current situation category is determined as: safe;

[0098] Exemplarily, when the first type of information obtained is: early morning, and the second type of information includes: snowy day, crossroads, and 40 km / s, it can be determined that the current situation is: a situation where traffic accidents such as collisions are likely to occur, and thus, the current situation category is determined as: easy to collide.

[0099] Among them, for each application scenario, it is possible to set various situations that may occur in the application scenario, and determine the first type of information and the second type of information matching each type of situation. In this way, after obtaining the first type of information and the second type of information, it is possible to determine the situation matching the obtained first type of information and the second type of information, and the category of the determined situation is the current situation category.

[0100] For example, the initial situation categories corresponding to the first type of information and the second type of information can be determined respectively according to the ranges to which the information values of the first type of information and the second type of information belong. Thus, among the two initial situation categories, the initial situation category with high energy consumption required by the second sensor is determined as the current situation category.

[0101] Exemplarily, in the vehicle collision avoidance scenario based on the Internet of Things, the possible situations that may occur in this application scenario can be set as follows: safe, difficult to collide, easy to collide, relatively easy to collide, and extremely easy to collide. Moreover, the first type of information and the second type of information matching each of the above situations are set. Suppose the first type of information is time, and the second type of information includes: weather, road conditions, and driving speed. Thus, when the Internet of Things middleware obtains the time, weather, road conditions, and driving speed at a certain moment, it can determine, from the above five situations of safe, difficult to collide, easy to collide, relatively easy to collide, and extremely easy to collide, the situation that matches the time, weather, road conditions, and driving speed obtained at that moment. Then, this situation is the situation at the current moment, and the category of this situation is the current situation category.

[0102] S103: Search, from the preset corresponding relationship between the situation category and the sampling frequency, for the sampling frequency corresponding to the current situation category as the target sampling frequency;

[0103] Since in order to reduce the energy consumption of each sensor as much as possible, for each situation, the sampling frequency corresponding to this situation can be set according to the requirements of this situation. Thus, a corresponding relationship between the situation category and the sampling frequency is established.

[0104] Among them, since the first type of sensor must work all the time, while the second type of sensor and the third type of sensor can work at a certain sampling frequency. Thus, the determined target sampling frequency is the sampling frequency to which the second type of sensor will be adjusted.

[0105] Optionally, after determining the above target sampling frequency, the specified sampling frequency to which the third type of sensor will be adjusted can also be determined according to this target sampling frequency. For example, the specified sampling frequency can be the same as the target sampling frequency. Or, for another example, the specified sampling frequency can be a multiple of the target sampling frequency, and this multiple can be any positive number, such as 0.5, 1, 2, etc., which are all reasonable.

[0106] Moreover, in the above corresponding relationship, for each situation category, for the situation category with a relatively urgent situation and thus a higher demand for information update, its corresponding sampling frequency is higher. Thus, in this situation, the energy consumption of the second type of sensor is higher.

[0107] Optionally, for the situation category with a relatively urgent situation and thus a higher demand for information update, when the determined target sampling frequency is higher, the specified sampling frequency used by the third sensor can also be increased accordingly. Thus, the energy consumption of the third type of sensor is also higher.

[0108] For example, in the vehicle collision avoidance scenario based on the Internet of Things, when there are five situations including safe, difficult to collide, easy to collide, relatively easy to collide, and extremely easy to collide, due to the arrangement order of safe, difficult to collide, easy to collide, relatively easy to collide, and extremely easy to collide, the situations of each situation become more and more urgent, and the demand for information update is higher and higher. Therefore, the sampling frequency corresponding to the safe situation is the lowest. Thus, in the safe situation, the energy consumption of the second type of sensor is the lowest. Optionally, the specified sampling frequency adopted by the third type of sensor can also be reduced, so that the energy consumption of the third type of sensor is the lowest; while the sampling frequency corresponding to the extremely easy to collide situation is the highest. Thus, in the safe situation, the energy consumption of the second type of sensor is the highest. Correspondingly, optionally, the specified sampling frequency adopted by the third type of sensor can also be increased, so that the energy consumption of the third type of sensor is the highest.

[0109] In this way, after determining the current situation category, the Internet of Things middleware can look up the sampling frequency corresponding to the current situation category from the pre-set correspondence between the situation category and the sampling frequency. Thus, the looked-up sampling frequency can be used as the target sampling frequency, and this target sampling frequency is the sampling frequency to which the second type of sensor needs to be adjusted. Optionally, the above-determined target sampling frequency can also be called the energy-saving and consumption-reducing scheme of the second type of sensor.

[0110] S104: Control the second type of sensor to collect information according to the target sampling frequency;

[0111] After determining the above target sampling frequency, the Internet of Things middleware can control the second type of sensor to collect information according to the target sampling frequency.

[0112] Optionally, the Internet of Things middleware can send a control instruction to the second type of sensor wirelessly to control the second type of sensor to collect information according to the target sampling frequency.

[0113] In the application scenario, the first type of sensor, the second type of sensor, and the third type of sensor corresponding to the application scenario can respectively collect information or determine information according to their respective collection frequencies.

[0114] Based on this, after the Internet of Things middleware controls the second type of sensor to collect information according to the target sampling frequency, at this time, the first type of sensor collects the first type of information in real time, the second type of sensor collects the second type of information according to the target sampling frequency, and correspondingly, the third type of sensor can also determine the third type of information according to the collection frequency set for it.

[0115] Among them, the so-called third type of information refers to information that continuously changes but does not require continuous determination. That is to say, for the third type of information determined by the third type of sensor, the third type of information continuously changes, but it does not require continuous determination and can be determined once every certain period of time. For example, in the vehicle collision avoidance scenario based on the Internet of Things, the third type of information can be the GPS information of the vehicle, and in the vehicle collision avoidance scenario based on the Internet of Things, the third type of sensor can be a GPS sensor.

[0116] Optionally, the acquisition frequency set for the third type of sensor can be set according to the actual situation of the application scenario and is independent of the first type of sensor and the second type of sensor.

[0117] Optionally, in order to further reduce the energy consumption of the third type of sensor during information acquisition, the acquisition frequency set for the third type of sensor can be less than the minimum acquisition frequency that the second type of acquisition frequency can adopt.

[0118] Optionally, in order to enable the acquisition frequency set for the third type of sensor to meet the needs of the currently determined context category, the acquisition frequency set for the third type of sensor can be determined based on the target sampling frequency. For example, the acquisition frequency set for the third type of sensor can be determined as the target sampling frequency, or the acquisition frequency set for the third type of sensor can be set as a preset multiple of the target sampling frequency, and the preset value can be any positive number. Exemplarily, the preset value can be 0.5, 1, 2, etc.

[0119] Based on this, optionally, in a specific implementation manner, a sensor control method based on context classification provided by an embodiment of the present invention may further include the following steps 11-12:

[0120] Step 11: Obtain a specified sampling frequency according to the target sampling frequency;

[0121] Among them, the specified sampling frequency is related to the current context category and has a multiple relationship with the target sampling frequency;

[0122] Step 12: Control the third type of sensor to determine the third type of information according to the specified acquisition frequency;

[0123] Among them, there are differences in the energy consumption of the first type of sensor, the second type of sensor, and the third type of sensor during information acquisition.

[0124] In this specific implementation manner, after determining the above target sampling frequency, when the Internet of Things middleware controls the second type of sensor to perform information acquisition according to the target sampling frequency, it can set a specified sampling frequency for the third type of sensor based on the determined target sampling frequency.

[0125] Among them, according to the requirements of the current situation category, a sampling frequency related to the current situation category and having a multiple relationship with the target sampling frequency can be set as the specified sampling frequency set for the third type of sensor. Furthermore, the third type of sensor can be controlled to determine the third type of information according to the specified sampling frequency.

[0126] Optionally, the IoT middleware can send a control instruction to the third type of sensor wirelessly to control the third type of sensor to collect information according to the specified sampling frequency.

[0127] Among them, for each situation category, for the situation category where the situation is relatively urgent and thus the demand for information update is relatively high, the multiple relationship between the specified sampling frequency and the target sampling frequency can be not less than 1, that is, the specified sampling frequency is greater than or equal to the target sampling frequency; while for the situation category where the situation is not urgent and thus the demand for information update is relatively low, the multiple relationship between the specified sampling frequency and the target sampling frequency can be not greater than 1, that is, the specified sampling frequency is less than or equal to the target sampling frequency.

[0128] Optionally, in a specific implementation manner, if the specified sampling frequency is the target sampling frequency, then step 12 above may include the following step 121:

[0129] Step 121: Control the third type of sensor to collect the third type of information according to the target sampling frequency.

[0130] In this specific implementation manner, the IoT middleware can control the third type of sensor to collect information according to the above target sampling frequency to obtain the third type of information.

[0131] Optionally, in a specific implementation manner, step 12 above may include the following step 122:

[0132] Step 122: Control the third type of sensor to infer the third type of information according to the target sampling frequency based on the information currently collected by the first type of sensor, the information currently collected by the second type of sensor, and the historical information determined by the third type of sensor.

[0133] In this specific implementation manner, the third type of information may not be obtained by the third type of sensor collecting information according to the above target sampling frequency, but is inferred according to the above target sampling frequency by using machine learning techniques.

[0134] Among them, the third type of sensor can, according to a machine learning algorithm, through model training, learn the corresponding relationship between the historical information it determines itself, the information collected by the first type of sensor, and the information collected by the second type of sensor. Thus, at each sampling moment, according to the above target sampling frequency, the information currently collected by the first type of sensor and the information currently collected by the second type of sensor can be obtained, and the information currently collected by the first type of sensor, the information currently collected by the second type of sensor, and the historical information determined by the third type of sensor are used to infer the third type of information at this sampling moment according to the learned corresponding relationship.

[0135] In this way, since the energy consumption of the third type of sensor is relatively high each time it collects information, therefore, using machine learning technology to infer the third type of information can reduce the energy consumption when the third type of sensor collects information, thereby further reducing the energy consumption of the third type of sensor.

[0136] Optionally, in a specific implementation manner, a sensor control method based on context classification provided by an embodiment of the present invention may further include the following steps 21-22:

[0137] Step 21: Search for the node mode corresponding to the current context category from the preset corresponding relationship between context categories and node modes as the target node mode;

[0138] Step 22: Control the second type of sensor to switch from the current node mode to the target node mode.

[0139] In this specific implementation manner, in different contexts, the second type of sensor has different sampling frequencies, and thus, the second type of sensor is in different energy consumption states, that is, in different contexts, the second type of sensor is in different node modes.

[0140] Based on this, for each context, the node mode corresponding to this context can be set, thereby establishing the corresponding relationship between context categories and node modes.

[0141] Among them, in the above corresponding relationship, for each context category, for the context category with a relatively urgent situation and thus a relatively high demand for information update, the corresponding node mode has a higher energy consumption.

[0142] For example, in the vehicle collision avoidance scenario based on the Internet of Things, when there are five contexts: safe, difficult to collide, easy to collide, relatively easy to collide, and extremely easy to collide, since in the order of safe, difficult to collide, easy to collide, relatively easy to collide, and extremely easy to collide, the situations of each context become more and more urgent and the demand for information update becomes higher and higher, therefore, the node modes corresponding to the above contexts of safe, difficult to collide, easy to collide, relatively easy to collide, and extremely easy to collide are: sleep, low energy consumption operation, medium-low energy consumption operation, medium-high energy consumption operation, and high energy consumption operation, respectively.

[0143] Obviously, the energy consumption of the node pattern corresponding to the safe scenario is the lowest. Thus, in the safe scenario, the energy consumption of the second type of sensor and the third type of sensor is the lowest, while the energy consumption of the node pattern corresponding to the extremely easy to collide scenario is the highest. Thus, in the safe scenario, the energy consumption of the second type of sensor and the third type of sensor is the highest.

[0144] In this way, after determining the current scenario category, the IoT middleware can search for the node pattern corresponding to the current scenario category from the pre-set correspondence between the scenario category and the node pattern. Thus, the found node pattern can be used as the target node pattern, which is the node pattern to which the second type of sensor needs to be adjusted.

[0145] Thus, after determining the above target node pattern, the IoT middleware can control the second type of sensor to switch from the current node pattern to the above target node pattern.

[0146] In addition, by combining the established correspondence between the scenario category and the sampling frequency, and the correspondence between the scenario category and the node pattern, the correspondence between the scenario category, the sampling frequency, and the node pattern can be obtained.

[0147] For example, as shown in Table 1, it is a schematic diagram of a correspondence between a scenario category, a sampling frequency, and a node pattern.

[0148] Table 1

[0149]

[0150]

[0151] Among them, the sampling frequency of the second type of sensor includes the sampling interval and the number of samplings. And the sampling interval is denoted as T i , and the number of samplings is denoted as N i , then the sampling period can be denoted as F i = T i N i (i = 1, 2, 3, 4, 5). Naturally, in the sleep mode, the data acquisition period of the sensing device is much longer than that of each working mode, approaching no sampling, that is, T 1 >> T 2 > T 3 > T 4 > T 5 , N 1 << N 2 < N 3 < N 4 < N 5The node modes include sleep, low power consumption operation, medium - low power consumption operation, medium - high power consumption operation, and high power consumption operation. Among them, the sampling interval, sampling times, and sampling period corresponding to different scenarios are defined according to the configuration file of the application scenario.

[0152] In this way, it is more flexible and user - friendly than the traditional fixed sampling frequency, and can achieve the purpose of energy conservation and consumption reduction.

[0153] Exemplarily, in the vehicle collision avoidance scenario based on the Internet of Things, Table 1 above can be transformed into Table 2:

[0154] Table 2

[0155] Situation Judgment Condition Node Mode <![CDATA[Sampling interval T i > <![CDATA[Sampling times N i > <![CDATA[Sampling period F i > Safety Weight Value 0.4 Sleep <![CDATA[T 1 > <![CDATA[N 1 > <![CDATA[T 1 N 1 > Difficult to Collide 0.5～0.7 Low Energy Consumption Operation <![CDATA[T 2 > <![CDATA[N 2 > <![CDATA[T 2 N 2 > Easy to Collide 0.8～1.1 Medium-Low Energy Consumption Operation <![CDATA[T 3 > <![CDATA[N 3 > <![CDATA[T 3 N 3 > Relatively Easy to Collide 1.2～1.5 Medium-High Energy Consumption Operation <![CDATA[T 4 > <![CDATA[N 4 > <![CDATA[T 4 N 4 > Extremely Easy to Collide 1.6 High Energy Consumption Operation <![CDATA[T 5 > <![CDATA[N 5 > <![CDATA[T 5 N 5 >

[0156] Optionally, in a specific implementation manner, based on the above - mentioned various specific implementation manners, a sensor control method based on context classification provided by an embodiment of the present invention may further include the following step 31:

[0157] Step 31: Start a preset response action corresponding to the current context category.

[0158] In this specific implementation manner, in some scenarios, in order to avoid accidents, ensure the safety of personnel and equipment, and enable the normal operation of the Internet of Things, it is necessary to control various physical devices in this scenario to take corresponding response actions to cope with this scenario.

[0159] For example, in the vehicle collision avoidance scenario based on the Internet of Things, when the current context category is extremely easy to collide, the Internet of Things middleware can start a preset collision avoidance action. Thus, start the collision avoidance device in the vehicle to prevent problems before they occur, and can start a preset positioning action. Thus, start the position sensor in the vehicle to accurately position the vehicle, so as to give an alarm in time when the vehicle is in danger.

[0160] Optionally, in a specific implementation manner, based on the above - mentioned various specific implementation manners, a sensor control method based on context classification provided by an embodiment of the present invention may further include the following step 41:

[0161] Step 41: Output the current context category and / or target node mode through the preset Web interface.

[0162] In this specific implementation manner, in order to facilitate users to remotely monitor the application scenario, a Web interface can be set on the Internet of Things middleware. Thus, the Internet of Things middleware can output the determined current context category and / or target node mode through this Web interface.

[0163] Based on this, when applying the solution provided by the embodiments of the present invention in the actual application of the Internet of Things, different sampling frequencies can be allocated to different sensors according to the requirements of the application scenario and the energy consumption of each sensor. Moreover, the sampling frequencies of each sensor can be adjusted according to the changes in the current situation, so that the sampling frequencies of each sensor can match the current situation category. In this way, through the combination of context awareness technology and the Internet of Things middleware, different sensors can be in different working modes, achieving the effect of reducing the energy consumption of each sensor in the Internet of Things, prolonging the service life of each sensor, reducing the maintenance requirements of each sensor, and lowering the maintenance cost.

[0164] Optionally, in a specific implementation manner, as Figure 2 shown, the above step S102 of determining the current situation category based on the first type of information and the second type of information may include the following steps S1021 - S1022:

[0165] S1021: Determine the target decision condition according to the first type of information and the second type of information;

[0166] wherein, the target decision condition is used to divide the current real environment into the corresponding situation state;

[0167] S1022: Search for the situation category corresponding to the target decision condition from the preset corresponding relationship between the decision condition and the situation category as the current situation category.

[0168] In this specific implementation manner, the first type of information collected by the first type of sensors and the second type of information collected by the second type of sensors in the application scenario can reflect the real environment currently presented in the application scenario, and further, can reflect the situation state corresponding to this real environment. That is to say, the first type of information collected by the first type of sensors and the second type of information collected by the second type of sensors in the application scenario can be used to perform situation classification on the current real environment in the application scenario, so as to divide the current real environment into the corresponding situation state.

[0169] For example, in the vehicle collision avoidance scenario based on the Internet of Things, the first type of sensor is a time sensor, and the second type of sensors include a meteorological sensor, a direction sensor, and a speed sensor. Further, when the first type of information obtained is: morning, and the second type of information includes: sunny, straight road, and 20 km / s, the above first type of information and the second type of information can reflect that the real environment currently presented in the vehicle collision avoidance scenario based on the Internet of Things is in the morning on a sunny day, with relatively high visibility, and the vehicle is driving at a low speed on a road with good road conditions. Thus, the above first type of information and the second type of information can reflect that the situation state corresponding to the above real environment is: a safe driving environment. In this way, the above real environment can be divided into the situation state of safety.

[0170] Based on this, for each possible situation in the application scenario, according to the real environment corresponding to this situation, the real first type of information and the second type of information in the situation category of this situation can be set. Furthermore, according to the real first type of information and the second type of information, the judgment conditions corresponding to the situation category of this situation can be set.

[0171] That is to say, for the application scenario, the various situation categories that may exist in this application scenario can be set first. Furthermore, according to the real environment corresponding to each situation category, the real first type of information and the second type of information in each situation category can be obtained. Thus, according to the real first type of information and the second type of information in each situation category, the judgment conditions corresponding to each situation category can be set. Furthermore, the corresponding relationship between the judgment conditions and the situation categories can be established.

[0172] In this way, after obtaining the first type of information and the second type of information, first, according to the comparison results between the obtained first type of information and the second type of information and the real first type of information and the second type of information in each situation category, the situation state corresponding to the current real environment can be determined. Furthermore, the current real environment can be divided into the corresponding situation state to determine the target judgment conditions for the current situation category. Furthermore, in the above corresponding relationship between the judgment conditions and the situation categories, the situation category corresponding to the target judgment conditions can be searched, and the determined situation category is the current situation category.

[0173] For example, as the first two columns in Table 1 above, it is a schematic diagram of a corresponding relationship between judgment conditions and situation categories. Exemplarily, as shown in Table 2, in the corresponding relationship between judgment conditions and situation categories in the vehicle collision avoidance scenario based on the Internet of Things, the judgment condition can be a weight value.

[0174] Also for example, for the various situation categories that may exist in the application scenario, according to the real environment corresponding to this situation category, the information value ranges to which the information values of the real first type of information and the second type of information in each situation category belong are obtained. In this way, the obtained various information value ranges can be used as judgment conditions. Thus, the corresponding relationship between the information value ranges and the situation categories is established, and this corresponding relationship is the corresponding relationship between the judgment conditions and the situation categories.

[0175] Exemplarily, as shown in Table 3, it is a schematic diagram of a corresponding relationship between judgment conditions and situation categories. Among them, the first column is the information value ranges to which the information values of the real first type of information and the second type of information in each situation category belong, and the second column is each situation category.

[0176] Table 3

[0177] Judgment Condition (Information Value Range) Situation Category Range 1 A Range 2 B Range 3 C Range 4 D Range 5 E

[0178] Based on the corresponding relationship between the judgment conditions and the situation categories shown in Table 3, the target information value range to which the obtained first type of information and second type of information belong can be first determined as the target judgment condition for characterizing the situation state corresponding to the current real environment. Furthermore, in Table 3 above, the situation category corresponding to this target information value range is searched for, so as to obtain the situation category corresponding to the target judgment condition as the current situation category.

[0179] Optionally, in a specific implementation manner, step S1021 of determining the target judgment condition according to the first type of information and the second type of information may include the following steps 41-42:

[0180] Step 41: Determine the respective first contribution degrees of each piece of the first type of information to forming the situation state corresponding to the current real environment, and the respective second contribution degrees of each piece of the second type of information to forming the above situation state;

[0181] In this specific implementation manner, under each situation category, each piece of the first type of information and each piece of the second type of information jointly form the situation state under this situation category. That is to say, for the formation of the situation state under this situation category, each piece of the first type of information and each piece of the second type of information have all made contributions and played roles.

[0182] Among them, for the formation of the situation state under this situation category, the roles played by different pieces of the first type of information and the second type of information may be the same or different, that is, for forming the situation state corresponding to the current real environment, the contribution degrees of each piece of the first type of information and each piece of the second type of information may be the same or different.

[0183] In this way, for each situation that may exist in the application scenario, according to the actual pieces of the first type of information and the second type of information in this situation, the contribution degrees of the information value pairs of each piece of the first type of information and each piece of the second type of information that match this situation to forming this situation can be respectively set. Thus, after the obtained pieces of the first type of information and the second type of information, the respective first contribution degrees of each piece of the first type of information to forming the situation state corresponding to the current real environment and the respective second contribution degrees of each piece of the second type of information to forming the situation state corresponding to the current real environment can be determined according to the information values of the obtained pieces of the first type of information and the second type of information.

[0184] Step 42: Based on the determined respective first contribution degrees and respective second contribution degrees, determine the target judgment condition for characterizing the situation state.

[0185] After determining the above-mentioned respective first contribution degrees and respective second contribution degrees, the target decision condition for characterizing the situation state corresponding to the current real environment can be determined based on the determined respective first contribution degrees and respective second contribution degrees.

[0186] Optionally, step 41 of determining the respective first contribution degrees of each piece of first-type information to forming the situation state corresponding to the current real environment and the respective second contribution degrees of each piece of second-type information to forming the above situation state may include the following step 51:

[0187] Step 51: Search for the weight values corresponding to each piece of first-type information and each piece of second-type information respectively from the preset correspondence between information values and weight values, and use them as the respective first contribution degrees and respective second contribution degrees.

[0188] For each situation that may exist in the application scenario, according to the contribution degrees of the real respective first-type information and respective second-type information in this situation to forming the situation state under this situation category, the weight values of the information values of the respective first-type information and respective second-type information that match this situation can be set respectively, as the contribution degrees of the respective first-type information and respective second-type information to forming the situation state under this situation category, that is, establish the correspondence between information values and weight values. In this way, after obtaining each piece of first-type information and each piece of second-type information, the weight values corresponding to each piece of first-type information and each piece of second-type information can be searched from the above-mentioned correspondence between information values and weight values, and used as the respective first contribution degrees and respective second contribution degrees.

[0189] For example, as shown in Table 4, it is a schematic diagram of a correspondence between information values and weight values. Among them, factor 1 to factor N are the information types of the first-type information and second-type information collected by each first-type sensor and second-type sensor, and conditional expressions 11 to conditional expressions nm are the specific information values of the first-type information collected by each first-type sensor and the second-type information collected by each second-type sensor. And for each conditional expression, the weight value in the same column as this conditional expression is the weight value corresponding to this information value, that is, the contribution degree of the first-type information or second-type information corresponding to this information value to forming various situation states.

[0190] In this way, after obtaining the above-mentioned various first-type information and various second-type information, the conditional expressions that match the information values of the various first-type information and the various second-type information can be searched in Table 4. Then, the weight values in the same columns as the found conditional expressions are the respective first contribution degrees of the various first-type information to the formation of the situational state corresponding to the current real environment, and the respective second contribution degrees of the various second-type information to the formation of the above situational state.

[0191] Table 4

[0192] Weight Value 0.1 0.2 ...... 0.9 Factor 1 Condition Expression 11 Condition Expression 12 ...... Condition Expression 1m Factor 2 Condition Expression 21 Condition Expression 22 ...... Condition Expression 2m ...... ...... ...... ...... ...... Factor n Condition Expression n1 Condition Expression n2 ...... Condition Expression nm

[0193] Among them, factors 1 to N can reflect the characteristic requirements of the application scenario. Moreover, the weight values of the n factors can all be normalized to 0.1 - 0.9. And the higher the corresponding weight value, the more likely this event is to occur.

[0194] Since various situations in the application scenario are determined based on the combination of various information and no definite value can be given, only a conditional expression within a certain range can be provided. Therefore, the fuzzy theory is used for the judgment of the situation, and many values can be adopted to represent the importance degree of this event, rather than the absolute 0 and 1 problems of the traditional judgment conditions.

[0195] Optionally, the various first-type information and the various second-type information are put into one-to-one correspondence with the conditional expressions, and then the weight values corresponding to the expressions are added to obtain the total weight. Thus, it can be matched with the various situations and various node patterns set in Table 1.

[0196] Exemplarily, in the vehicle collision avoidance scenario based on the Internet of Things, the above Table 4 can be transformed into Table 5:

[0197] Table 5

[0198] Weight Value 0.1 0.2 0.3 0.4 Traffic Time Night, Early Morning Morning, Afternoon Noon Morning, Evening Weather Sunny, Cloudy Overcast, Light Rain Moderate to Heavy Rain, Snow Thunderstorm, Snow Road Condition Straight Road Small Straight Road Curve Crossroads Driving Speed 0 - 30 km / h 30 - 50 km / h 50 - 80 km / h > 80 km / h

[0199] Correspondingly, optionally, on the basis of the above step 51, in the above step 42, based on the determined respective first contribution degrees and respective second contribution degrees, to determine the target judgment condition for characterizing the situational state, the following step 52 may be included:

[0200] Step 52: Calculate the sum value of the determined respective first contribution degrees and respective second contribution degrees as the target judgment condition for characterizing the situational state.

[0201] When using the weight values corresponding to the information values of each piece of first - type information and each piece of second - type information as the respective first contribution degrees of each piece of first - type information to the formation of the situation state corresponding to the current real environment and the respective second contribution degrees of each piece of second - type information to the formation of the above - mentioned situation state, the sum value of the determined respective first contribution degrees and respective second contribution degrees can be calculated, that is, calculate the sum value of the weight values corresponding to the information values of each piece of first - type information and each piece of second - type information. Thus, this sum value can be used as the target decision condition for characterizing the situation state.

[0202] That is to say, the decision conditions in Table 1 can be the respective intervals where the sum value of the weight values is located.

[0203] In addition, optionally, based on the above step 51, in step 42, determining the target decision condition for characterizing the situation state based on the determined respective first contribution degrees and respective second contribution degrees may include the following step 53:

[0204] Step 53: Calculate the average value of the determined respective first contribution degrees and respective second contribution degrees as the target decision condition for characterizing the situation state

[0205] When using the weight values corresponding to the information values of each piece of first - type information and each piece of second - type information as the respective first contribution degrees of each piece of first - type information to the formation of the situation state corresponding to the current real environment and the respective second contribution degrees of each piece of second - type information to the formation of the above - mentioned situation state, the average value of the determined respective first contribution degrees and respective second contribution degrees can be calculated, that is, calculate the average value of the weight values corresponding to the information values of each piece of first - type information and each piece of second - type information. Thus, this average value can be used as the target decision condition for characterizing the situation state.

[0206] That is to say, the decision conditions in Table 1 can be the respective intervals where the average value of the weight values is located.

[0207] Optionally, in a specific implementation manner, as Figure 3 shown, a sensor control method based on situation classification provided by an embodiment of the present invention may further include the following step S105:

[0208] S105: Perform data pre - processing on the first - type information and / or the second - type information to obtain the pre - processed first - type information and / or second - type information;

[0209] Correspondingly, in this specific implementation manner, in the above step S102, determining the current situation category based on the first - type information and the second - type information may include the following steps S102A - S102B:

[0210] S102A: In the case of performing data preprocessing on the first type of information or the second type of information, based on the preprocessed first type of information and the second type of information, determine the current situation category, or, based on the first type of information and the preprocessed second type of information, determine the current situation category;

[0211] S102B: In the case of performing data preprocessing on the first type of information and the second type of information, based on the preprocessed first type of information and the preprocessed second type of information, determine the current situation category.

[0212] In this specific implementation, since the first type of sensor and the second type of sensor collect information independently, therefore, in order to provide relatively reliable first type of information and second type of information, after obtaining the above-mentioned first type of information and second type of information, the IoT middleware can first perform data preprocessing on the first type of information and / or the second type of information.

[0213] That is to say, data preprocessing can be performed only on the first type of information, or only on the second type of information, or data preprocessing can be performed on both the first type of information and the second type of information.

[0214] Optionally, the above data preprocessing can include at least one of the following various processing methods:

[0215] Data cleaning processing, data integration processing, and data transformation processing.

[0216] Among them, the so-called data cleaning processing can include: missing value processing, for example, using methods such as data imputation to insert and supplement the missing values in the data; noise processing, for example, using methods such as binning, clustering, and regression to remove the noise in the data;

[0217] The so-called data integration processing can include: entity recognition, redundancy and correlation analysis, and numerical conflict monitoring and processing;

[0218] The so-called data transformation processing can include: data normalization, data discretization, and concept hierarchy.

[0219] Among them, the methods included in the above various data preprocessings are only examples of various data preprocessings, rather than limitations. Of course, the above data preprocessing can also include other processing methods, and in this regard, the embodiments of the present invention do not make specific actions.

[0220] In this way, after the data preprocessing is completed, the preprocessed first type of information and / or the preprocessed second type of information can be obtained.

[0221] Furthermore, in the case of only performing data preprocessing on the first type of information, the current situation category can be determined based on the preprocessed first type of information and the obtained second type of information;

[0222] In the case of only performing data preprocessing on the second type of information, the current situation category can be determined based on the obtained first type of information and the preprocessed second type of information;

[0223] In the case of performing data preprocessing on both the first type of information and the second type of information, the current situation category can be determined based on the preprocessed first type of information and the preprocessed second type of information.

[0224] Among them, the specific implementation manners of the above steps S102A - S102B are similar to the specific implementation manner of the above step S102, and will not be elaborated here.

[0225] Optionally, in a specific implementation manner, based on the above various specific implementation manners, a sensor control method based on situation classification provided by an embodiment of the present invention may further include the following step 71:

[0226] Step 71: Determine whether the current situation category matches a preset topic subscription table; if it matches, execute step S104 to control the second type of sensor to collect information at a target sampling frequency, and control the third type of sensor to determine the third type of information at the target sampling frequency.

[0227] Among them, the topic subscription table is set based on the subscription and publishing services provided by the IoT middleware;

[0228] In this specific implementation manner, the IoT middleware may be provided with a subscription and publishing mechanism, so that the IoT middleware can provide subscription and publishing services.

[0229] The so-called subscription and publishing mechanism is a mechanism for real-time data message distribution based on context themes. Furthermore, the subscription and publishing services provided by the IoT middleware may include one or more distributed message brokers. Among them, after the message producer publishes a message, the message broker can match the message published by the message producer with a preset topic subscription table. Thus, the message is passed to the consumers who have subscribed to the message, that is, the broker executes routing the message from the message producer to the message subscriber. Among them, the message broker needs to maintain two types of information: the message set and the topic subscription table. The message set stores all the latest messages related to each topic. Message routing processes the incoming messages according to the topic subscription table and routes the incoming messages to the message subscribers who have subscribed to the message. If the message subscriber is only managed by this message broker, the message broker can directly notify the message subscriber of the message through the topic notification interface. If the message subscriber is not registered in this message broker, the message will be routed by this message broker to other relevant brokers. In order to maintain the consistency of subscriptions in the data distribution service, all message brokers need to synchronize subscription information according to the message routing policy.

[0230] In this way, in some application scenarios, users may only care about the operation of the IoT in some of the contexts and do not need to pay attention to all the contexts in this application scenario. Thus, based on the subscription and publishing services provided by the IoT middleware, users can set a topic subscription table. Among them, the topic subscription table includes various context categories that the user hopes to subscribe to.

[0231] That is to say, based on the subscription and publishing services provided by the IoT middleware, users can set various context categories that they hope to subscribe to, and the various context categories that the user hopes to subscribe to are recorded in the topic subscription table.

[0232] Thus, after determining the current context category, the IoT middleware can determine whether the current context category matches the preset topic subscription table, that is, whether the current context category exists in the preset topic subscription table.

[0233] Among them, if there is a match, it means the current context category that the user hopes to subscribe to. Furthermore, control the second type of sensor to collect information at the target sampling frequency and control the third type of sensor to determine the third type of information at the target sampling frequency.

[0234] Correspondingly, if there is no match, it means the current context category that the user does not hope to subscribe to, then there is no need to control the second type of sensor to collect information at the target sampling frequency, nor is it necessary to control the third type of sensor to determine the third type of information at the target sampling frequency.

[0235] Based on the sensor control method based on context classification provided in the embodiments of the present invention above, the embodiments of the present invention also provide an Internet of Things middleware.

[0236] Figure 4 The following is a schematic structural diagram of an Internet of Things middleware provided in an embodiment of the present invention. As Figure 4 shown, the Internet of Things middleware includes: a context awareness module 410, an open service interface module 420, and a comprehensive application platform 430;

[0237] Among them, the context awareness module 410 is used to: obtain the first type of information collected in real time by the first type of sensors and the second type of information collected by the second type of sensors according to the current sampling frequency; and determine the current context category based on the first type of information and the second type of information; search for the sampling frequency corresponding to the current context category from the preset correspondence between context categories and sampling frequencies as the target sampling frequency; and send the target sampling frequency to the comprehensive application platform 430 through the open service interface module 420; wherein, there are differences in the energy consumption of the first type of sensors and the second type of sensors during the information collection process;

[0238] The comprehensive application platform 430 is used to: control the second type of sensors to collect information according to the target sampling frequency.

[0239] In the embodiments of the present invention, the Internet of Things middleware includes: a context awareness module 410, an open service interface module 420, and a comprehensive application platform 430. Among them, the context awareness module 410 is used to determine the target sampling frequency, that is, determine the energy-saving and consumption-reducing scheme of the second type of sensors, by using the above-mentioned first type of information and the second type of information; and the comprehensive application platform 430 is used to control the second type of sensors to execute the above-mentioned energy-saving and consumption-reducing scheme. Therefore, an open service interface module 420 can be set up to provide an interface to realize the information transmission between the context awareness module 410 and the comprehensive application platform 430.

[0240] Among them, the so-called open service interface module can provide a RESTful API (Representational State Transfer-based application programming interface) to obtain the information collected by the sensors set at the bottom layer of the Internet of Things, and can send control commands to the lower layer on the Internet. Among them, REST is the abbreviation of Representational State Transfer, and its Chinese meaning is: Representational State Transfer, and API is the abbreviation of Application Programming Interface, and its Chinese meaning is: Application Programming Interface.

[0241] Specifically, REST, also known as Representational State Transfer, is an architecture style oriented towards resources. In this architecture style, each sensor node is connected to multiple sensors or actuators. All kinds of resources, including the sensor itself and the resources provided through the network, can be connected to the Web service using their own URL (Uniform Resource Locator). The RESTful API can adopt predefined functions, locate resources using the URL, and describe operations with HTTP (Hyper Text Transfer Protocol) verbs, providing a lightweight interface for service components, enabling users to easily use them to call their functions. Moreover, five abstract functions can be defined to package Internet of Things resources into the RESTful API. These abstract functions can include controlling the list of available devices, accessing the current settings of a device, retrieving the list of devices controlled by a given device, retrieving the list of available commands of a given device, and executing measures.

[0242] That is to say, in the embodiment of the present invention, the open service interface module 420 can provide a RESTful API interface, enabling the context awareness module 410 to send the target sampling frequency to the integrated application platform 430 through this RESTful API interface.

[0243] Optionally, the integrated application platform 430 can wirelessly send control instructions to the second type of sensors to control the second type of sensors to collect information at the target sampling frequency.

[0244] Based on this, by applying the solution provided in the embodiment of the present invention, in the actual application of the Internet of Things, different sampling frequencies can be assigned to different sensors according to the requirements of the application scenario and the energy consumption of each sensor. Moreover, the sampling frequencies of each sensor can be adjusted according to the change of the current situation, so that the sampling frequencies of each sensor can match the current situation category. In this way, through the combination of context awareness technology and the Internet of Things middleware, different sensors can be in different working modes, achieving the effect of reducing the energy consumption of each sensor in the Internet of Things, extending the service life of each sensor, reducing the maintenance requirements of each sensor, and lowering the maintenance cost.

[0245] Optionally, in a specific implementation manner, the above integrated application platform 430 is further configured to:

[0246] Obtain a specified sampling frequency according to the target sampling frequency; where the specified sampling frequency is related to the current situation category and has a multiple relationship with the target sampling frequency;

[0247] Control the third type of sensors to determine the third type of information according to a specified sampling frequency; among them, the energy consumption of the first type of sensors, the second type of sensors, and the third type of sensors is different during the information acquisition process.

[0248] Optionally, in a specific implementation manner, the above-mentioned context awareness module 410 is further configured to:

[0249] Perform data preprocessing on the first type of information and / or the second type of information to obtain the preprocessed first type of information and / or the second type of information;

[0250] Correspondingly, in this specific implementation manner, the above-mentioned context awareness module 410 determines the current context category based on the first type of information and the second type of information, that is, it may include:

[0251] In the case of performing data preprocessing on the first type of information or the second type of information, the context awareness module 410 determines the current context category based on the preprocessed first type of information and the second type of information, or the context awareness module 410 determines the current context category based on the first type of information and the preprocessed second type of information;

[0252] In the case of performing data preprocessing on the first type of information and the second type of information, the context awareness module 410 determines the current context category based on the preprocessed first type of information and the preprocessed second type of information.

[0253] Optionally, in a specific implementation manner, before the context awareness module 410 sends the target sampling frequency to the comprehensive application platform through the open service interface module, the context awareness module 410 is further configured to:

[0254] Determine whether the current context category matches the preset topic subscription table; where the preset topic subscription table is set based on the subscription and publishing services provided by the context awareness module;

[0255] If it matches, send the target sampling frequency to the comprehensive application platform 430 through the open service interface module 420.

[0256] Optionally, in a specific implementation manner, the context awareness module 410 is further configured to:

[0257] Search for the node mode corresponding to the current context category from the preset correspondence between the context category and the node mode as the target node mode; and send the target node mode to the comprehensive application platform 430 through the open service interface module 420;

[0258] The comprehensive application platform 430 is further configured to control the second type of sensors to switch from the current node mode to the target node mode.

[0259] In this specific implementation, after determining the target node mode, the context awareness module 410 can also send the target node mode to the integrated application platform 430 through the interface provided by the open service interface module 420. Thus, the integrated application platform 430 can control the second type of sensor to switch from the current node mode to the target node mode.

[0260] Optionally, in a specific implementation, the context awareness module 410 is further configured to:

[0261] Send the current context category to the integrated application platform 430 through the open service interface module 420;

[0262] The integrated application platform 430 is further configured to start a preset response action corresponding to the current context category.

[0263] In this specific implementation, the current context category is determined by the context awareness module 4101. Thus, the context awareness module 410 can send the current context category to the integrated application platform 430 through the interface provided by the open service interface module 420. Thus, the integrated application platform 430 can control the second type of sensor to switch from the current node mode to the target node mode.

[0264] Optionally, in a specific implementation, the integrated application platform 430 is further configured to:

[0265] Output the current context category and / or the target node mode through the preset Web interface.

[0266] In this specific implementation, the integrated application platform 430 is provided with a Web interface. Then, the integrated application platform 430 can output the received current context category and / or target node mode through the Web interface.

[0267] Optionally, in a specific implementation, the context awareness module 410 includes: a data preprocessing unit and a decision maker;

[0268] The data preprocessing unit is configured to perform data preprocessing on the first type of information and / or the second type of information to obtain the preprocessed first type of information and / or the second type of information;

[0269] A decider, which is used to determine the current situation category based on the preprocessed first type of information and the second type of information when performing data preprocessing on the first type of information or the second type of information, or to determine the current situation category based on the first type of information and the preprocessed second type of information; when performing data preprocessing on the first type of information and the second type of information, determine the current situation category based on the preprocessed first type of information and the preprocessed second type of information; and search for the sampling frequency corresponding to the current situation category from the preset correspondence between the situation category and the sampling frequency as the target sampling frequency.

[0270] In this specific implementation manner, the situation awareness module 410 may further include a data preprocessing unit and a decider. Thus, the data preprocessing unit can be used to: obtain the first type of information collected in real time by the first type of sensors and the second type of information collected by the second type of sensors at the current sampling frequency; perform data preprocessing on the first type of information and / or the second type of information to obtain the preprocessed first type of information and / or the second type of information.

[0271] Furthermore, when only performing data preprocessing on the first type of information, the data preprocessing unit can send the preprocessed first type of information and the obtained second type of information to the decider. Thus, the decider can determine the current situation category based on the preprocessed first type of information and the second type of information obtained by the data preprocessing unit.

[0272] When only performing data preprocessing on the second type of information, the data preprocessing unit can send the obtained first type of information and the preprocessed second type of information to the decider. Thus, the decider can determine the current situation category based on the first type of information obtained by the data preprocessing unit and the preprocessed second type of information.

[0273] When performing data preprocessing on the first type of information and the second type of information, the data preprocessing unit can send the preprocessed first type of information and the preprocessed second type of information to the decider. Thus, the decider can determine the current situation category based on the preprocessed first type of information and the preprocessed second type of information.

[0274] After determining the current situation category, the decider can search for the sampling frequency corresponding to the current situation category from the preset correspondence between the situation category and the sampling frequency as the target sampling frequency. And further send the target sampling frequency to the comprehensive application platform 430 through the open service interface module 420.

[0275] To facilitate the understanding of a sensor control method based on scenario classification and an IoT middleware provided in the above embodiments of the present invention, the following will give an example through the application of the above method and IoT middleware in a vehicle collision avoidance scenario based on the Internet of Things.

[0276] As Figure 5 shown, it is an architecture diagram of an IoT middleware provided in the embodiments of the present invention in a vehicle collision avoidance scenario based on the Internet of Things.

[0277] Among them, the so-called environment is the real environment in the vehicle collision avoidance scenario based on the Internet of Things; the so-called wireless sensor network is composed of the first type of sensor, the second type of sensor, and the third type of sensor, and is used to sense the situation corresponding to the real environment; the so-called device is various hardware devices in the real environment in the vehicle collision avoidance scenario based on the Internet of Things except for the wireless sensor network. In addition, the IoT middleware includes: a situation awareness module, an open service interface module, and a comprehensive application platform, and the situation awareness module includes: a data preprocessing module and a decision maker, and a subscription and publishing mechanism is set.

[0278] Specifically: First, according to the vehicle position collected by the GPS sensor, it can be determined that the vehicle is on the road. Furthermore, it can be determined that the current scenario is a vehicle collision avoidance scenario based on the Internet of Things. Thus, among the pre-imported various sensors, the combination of the first type of sensor, the second type of sensor, and the third type of sensor that matches the vehicle collision avoidance scenario based on the Internet of Things can be determined.

[0279] Among them, the first type of sensor can be a time sensor, which is used to collect traffic time information. The second type of sensor is a meteorological sensor, a direction sensor, and a speed sensor, which are respectively used to collect weather information, road condition information, and driving speed information. The third type of sensor is a GPS sensor, which is used to collect position information.

[0280] Furthermore, the data preprocessing module can be connected to the wireless sensor network in the scenario, so that it can obtain the traffic time information collected in real time by the time sensor, as well as the weather information, road condition information, and driving speed information collected by the meteorological sensor, direction sensor, and speed sensor respectively according to the current sampling frequency. And the data preprocessing module can perform data preprocessing on the collected traffic time information, weather information, road condition information, and driving speed information, for example, data cleaning processing and data transformation processing, so as to obtain the processed traffic time information, weather information, road condition information, and driving speed information.

[0281] After that, the processed traffic time information, weather information, road condition information, and driving speed information can be sent to the decision maker. Through Table 5 above, the current situation category is determined. Furthermore, through Table 2 above, the sampling frequency corresponding to the current situation category is determined to obtain the target sampling frequency, and the node mode corresponding to the current situation category is determined to obtain the target node mode.

[0282] For example, assume that the IoT middleware is set in a certain vehicle. It is evening and it is raining moderately. The vehicle is driving on a small straight road at a speed of 40 km / h. Thus, the time sensor can collect traffic time information, and the meteorological sensor, direction sensor, and speed sensor can respectively collect weather information, road condition information, and driving speed information, and the information set Iv=(evening, moderate rain, small straight road, 40) can be obtained; according to Table 5, the sum of the corresponding weight values is: 0.4 + 0.3 + 0.2 + 0.2 = 1.1; furthermore, according to Table 2, the current situation category can be obtained as easy to collide. Thus, the target node mode can be determined as: medium and low energy consumption working mode, and the target sampling frequency can be determined as: sampling interval T3, sampling times N3, and sampling period T3N3.

[0283] Furthermore, since the preset topic subscription table set by the user based on the subscription and publishing services provided by the subscription and publishing mechanism includes the above current situation category, thus, the above target node mode and target sampling frequency can be sent to the integrated application platform through the RESTful API provided by the open service interface module.

[0284] In this way, the integrated application platform can send control instructions to the meteorological sensor, direction sensor, speed sensor, and GPS sensor wirelessly to control the meteorological sensor, direction sensor, and speed sensor to collect weather information, road condition information, and driving speed information respectively according to the above target sampling frequency, and control the GPS sensor to determine new location information according to the above target sampling frequency using the above traffic time information, weather information, road condition information, driving speed information, and the historical location information determined by the GPS sensor.

[0285] In addition, the integrated application platform can also activate the pre-set anti-collision device to prevent problems before they occur, and can also start the vehicle sensing device to accurately locate the vehicle.

[0286] In another embodiment provided by the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the sensor control methods based on situation classification provided by the above embodiments of the present invention are implemented.

[0287] In another embodiment provided by the present invention, there is also provided a computer program product including instructions, which, when running on a computer, cause the computer to execute the steps of any one of the sensor control methods based on scenario classification provided in the embodiments of the present invention described above.

[0288] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0289] It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device including the element.

[0290] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the Internet of Things middleware, the computer-readable storage medium, and the computer program product, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0291] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included in the protection scope of the present invention.

Claims

1. A sensor control method based on context classification, characterized in that, applied to the IoT middleware, the method includes: Obtaining the first type of information collected in real time by the first type of sensors, and the second type of information collected by the second type of sensors according to the current sampling frequency; wherein, there are differences in energy consumption during the information collection process between the first type of sensors and the second type of sensors; Based on the first type of information and the second type of information, determining the current context category; Searching from the preset correspondence between context categories and sampling frequencies for the sampling frequency corresponding to the current context category as the target sampling frequency; Controlling the second type of sensors to collect information according to the target sampling frequency; wherein, the determining the current context category based on the first type of information and the second type of information includes: determining a target decision condition according to the first type of information and the second type of information; wherein, the target decision condition is used to divide the current real environment into corresponding context states; searching from the preset correspondence between decision conditions and context categories for the context category corresponding to the target decision condition as the current context category; or, the method further includes: obtaining a specified sampling frequency according to the target sampling frequency; wherein, the specified sampling frequency is related to the current context category and has a multiple relationship with the target sampling frequency; controlling the third type of sensors to collect the third type of information according to the specified sampling frequency; wherein, there are differences in energy consumption during the information collection process between the first type of sensors, the second type of sensors and the third type of sensors.

2. The method according to claim 1, characterized in that, before the determining the current context category based on the first type of information and the second type of information, the method includes: Performing data preprocessing on the first type of information and / or the second type of information to obtain the preprocessed first type of information and / or second type of information; the determining the current context category based on the first type of information and the second type of information includes: in the case of performing data preprocessing on the first type of information or the second type of information, determining the current context category based on the preprocessed first type of information and the second type of information, or, based on the first type of information and the preprocessed second type of information; in the case of performing data preprocessing on the first type of information and the second type of information, determining the current context category based on the preprocessed first type of information and the preprocessed second type of information.

3. The method according to claim 1, characterized in that, before the controlling the second type of sensors to collect information according to the target sampling frequency, the method further includes: Judging whether the current context category matches a preset topic subscription table; wherein, the topic subscription table is set based on the subscription and publishing services provided by the IoT middleware; if it matches, then controlling the second type of sensors to collect information according to the target sampling frequency.

4. The method according to claim 1, characterized in that, Determining a target decision condition according to the first type of information and the second type of information includes: Determining respective first contribution degrees of each piece of the first type of information to forming a situation state corresponding to the current real environment, and respective second contribution degrees of each piece of the second type of information to forming the situation state; Based on the determined respective first contribution degrees and respective second contribution degrees, determining a target decision condition for characterizing the situation state.

5. The method according to any one of claims 1-4, wherein, The specified sampling frequency is: the target sampling frequency; Controlling the third type of sensor to determine the third type of information according to the specified sampling frequency includes: Controlling the third type of sensor to collect the third type of information according to the target sampling frequency; or, Controlling the third type of sensor to infer the third type of information according to the target sampling frequency, based on the information currently collected by the first type of sensor, the information currently collected by the second type of sensor, and the historical information determined by the third type of sensor.

6. The method according to claim 1, wherein, The method further includes: Searching, from a preset correspondence between situation categories and node modes, for a node mode corresponding to the current situation category as a target node mode; Controlling the second type of sensor to switch from the current node mode to the target node mode.

7. An Internet of Things middleware, wherein, The Internet of Things middleware includes: a situation awareness module, an open service interface module, and a comprehensive application platform; The situation awareness module is configured to: Obtain the first type of information collected in real time by the first type of sensor and the second type of information collected by the second type of sensor according to the current sampling frequency; and determine the current situation category based on the first type of information and the second type of information; wherein, there are differences in energy consumption during the information collection process between the first type of sensor and the second type of sensor; Search, from a preset correspondence between situation categories and sampling frequencies, for a sampling frequency corresponding to the current situation category as a target sampling frequency; and send the target sampling frequency to the comprehensive application platform through the open service interface module; The comprehensive application platform is configured to: Control the second type of sensor to collect information according to the target sampling frequency; wherein, The situation awareness module is specifically configured to: determine a target decision condition according to the first type of information and the second type of information; wherein, the target decision condition is used to divide the current real environment into corresponding situation states; search, from a preset correspondence between decision conditions and situation categories, for a situation category corresponding to the target decision condition as the current situation category; or, The comprehensive application platform is further configured to: obtain a specified sampling frequency according to the target sampling frequency; wherein, the specified sampling frequency is related to the current situation category and has a multiple relationship with the target sampling frequency; control the third type of sensor to determine the third type of information according to the specified sampling frequency; wherein, there are differences in the energy consumption of the first type of sensor, the second type of sensor, and the third type of sensor during the information collection process.

8. The IoT middleware according to claim 7, wherein, before sending the target sampling frequency to the comprehensive application platform through the open service interface module, the situation awareness module is further configured to: determine whether the current situation category matches a preset topic subscription table; wherein, the preset topic subscription table is set based on the subscription and publishing services provided by the situation awareness module; if there is a match, send the target sampling frequency to the comprehensive application platform through the open service interface module.

9. A computer-readable storage medium, wherein, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1-6 are implemented.

Images