System and method for ensemble learning

Abstract

A system for ensemble learning is provided. The system includes a plurality of slave computing devices and a master computing device. Each slave computing device is configured to obtain a classification result by a weak classifier. The master computing device is configured to receive the classification results and output a final result based on the classification results.

Description

Technical Field

[0001] This invention relates to machine learning, and more particularly to ensemble learning systems and methods. Prior Technology

[0002] Edge computing is a distributed computing architecture that processes data at the edge device, close to the data source, without transmitting data to a data center or cloud computing platform. This avoids latency during data transmission and enhances real-time processing capabilities. These characteristics are crucial in many applications requiring instant decision-making, such as autonomous vehicles, smart cities, and industrial automation.

[0003] Common sources of edge computing data include sensors, IoT devices, and smart devices, which typically generate substantial amounts of data. However, because edge devices usually have limited computing and storage resources, they are less suited to performing computationally complex operations (such as inferences in complex machine learning models) or processing large amounts of data (such as training machine learning models).

[0004] Generally, the computations or processing that are difficult to perform on edge devices are completed in data centers or cloud computing platforms. The edge device then continues its operations based on the computation or processing results returned by the data center or cloud computing platform. For example, after training a simple model on the cloud computing platform, the trained simple model is deployed to the edge device. Or, for instance, after inferring a complex model on the cloud computing platform, the inference results are transmitted to the edge device. However, the above approach is no longer sufficient for today's complex and performance-critical application scenarios, such as speech recognition, image classification, and autonomous driving.

[0005] Therefore, an integrated learning system and method are needed to solve the above problems. Summary of the Invention

[0006] The embodiments disclosed herein provide an integrated learning system including complex slave computing units and a master computing unit. Each slave computing unit is configured to obtain a classification result using a weak classifier. The master computing unit is configured to receive classification results from the slave computing units and output a final classification result based on the classification results.

[0007] In one embodiment, each slave computing device is further configured to, during the training phase, receive first sensing data and train a weak classifier using the first sensing data; and during the inference phase, receive second sensing data and obtain a classification result for the second sensing data using the weak classifier. The master computing device is configured to, during the inference phase, receive the classification result from the slave computing device; and, based on the classification result, output the final classification result.

[0008] In one embodiment, each slave computing device is further configured to, during the training phase, share a portion of the first sensing data with other slave computing devices and receive the shared portions from other slave computing devices; and train a weak classifier using the first sensing data and the shared portions. The first sensing data of each slave computing device has the same data format.

[0009] In one embodiment, the main computing device is further configured to take the mode of the classification results as the final classification result during the inference stage.

[0010] In one embodiment, each weak classifier from the computing device is a decision tree model.

[0011] The embodiments disclosed herein provide an ensemble learning method executed in a computer system. The computer system includes a plurality of slave computing devices and a main computing device. The ensemble learning method includes obtaining classification results using a weak classifier through each slave computing device; and receiving classification results from the slave computing devices through the main computing device, and outputting a final classification result based on the classification results.

[0012] The embodiments disclosed herein provide an integrated learning system including a complex set of slave computing devices. Each slave computing device is configured to share a weak classifier with other slave computing devices and to receive other weak classifiers from other slave computing devices. Each slave computing device is further configured to use the weak classifiers and other weak classifiers to obtain a final classification result.

[0013] In one embodiment, each slave computing device is further configured to, during the training phase, receive first sensing data; train a weak classifier using the first sensing data; and share the weak classifier with other slave computing devices, and receive other weak classifiers from other slave computing devices. Each slave computing device is further configured to, during the inference phase, receive second sensing data and classify the second sensing data using the weak classifier and other weak classifiers to obtain a final classification result. The first sensing data of each slave computing device has the same data format.

[0014] In one embodiment, each slave computing device is further configured to, during the inference phase, input second sensing data to the weak classifier and other weak classifiers respectively to obtain individual classification results; and based on the classification results, output the final classification result.

[0015] In one embodiment, each slave computing device is further configured to, during the inference phase, take the mode of the classification results as the final classification result.

[0016] In one embodiment, each slave computing device is further configured to, during the training phase, share a portion of the first sensing data with other slave computing devices and receive the shared portions from other slave computing devices; and train a weak classifier using the first sensing data and the shared portions.

[0017] In one embodiment, each weak classifier from the computing device is a decision tree model.

[0018] In one embodiment, the ensemble learning system further includes a master computing unit. The master computing unit is configured to, during the inference phase, evaluate whether the performance of each slave computing unit's weak classifier exceeds a specified threshold; and, in response to a weak classifier's performance not exceeding the specified threshold, disallow slave computing units from sharing the weak classifier with other slave computing units.

[0019] The embodiments disclosed herein provide an ensemble learning method executed in a computer system, the computer system including a plurality of slave computing devices. The ensemble learning method includes sharing a weak classifier to other slave computing devices through each slave computing device, and receiving other weak classifiers from other slave computing devices. The ensemble learning method further includes classifying second sensed data through each slave computing device using the weak classifier and other weak classifiers to obtain final classified data.

[0020] The ensemble learning system and method disclosed herein enable ensemble learning training and inference on edge computing systems. Furthermore, by training weak classifiers on individual edge devices and deploying the strong classifier, formed by merging these weak classifiers, across the entire edge computing system or on individual edge devices, the accuracy of edge-side inference can be improved, while the latency caused by data transmission to the cloud can be reduced, thereby shortening model training time and enhancing real-time inference capabilities. Simple Explanation of the Diagram

[0021] This disclosure will be better understood from the following description of exemplary embodiments in conjunction with the accompanying diagrams. Furthermore, it should be understood that the execution order of the blocks in the flowcharts of this disclosure may be changed, and / or certain blocks may be modified, deleted, or merged. Figure 1 is a system architecture diagram of an integrated learning system according to an embodiment of the present invention. Figure 2A is a data flow diagram of the training phase of the ensemble learning method according to an embodiment of the present invention. Figure 2B is a data flow diagram of the inference phase of the ensemble learning method according to an embodiment of the present invention. Figure 3 is a system architecture diagram of an integrated learning system according to an embodiment of the present invention. Figure 4A is a data flow diagram of the training phase of an ensemble learning method according to an embodiment of the present invention. Figure 4B is a schematic diagram of the slave computing device that completes the training phase in this embodiment. Figure 4C is a data flow diagram of the inference phase of the ensemble learning method according to an embodiment of the present invention. Figure 5 is a schematic diagram illustrating the implementation of step S422 of Figure 4C executed from the computing device according to an embodiment of the present invention. Implementation

[0022] The following description illustrates various embodiments of the present invention, but is not intended to limit the scope of the invention. The actual scope of the invention is defined by the claims.

[0023] In the embodiments listed below, the same or similar elements or components will be represented by the same reference numerals.

[0024] The serial numbers in this specification and in the claims, such as "first," "second," etc., are for ease of explanation only and there is no sequential relationship between them.

[0025] The descriptions of embodiments of the apparatus or system in this specification also apply to embodiments of the method, and vice versa.

[0026] The embodiments disclosed herein provide an integrated learning system. This integrated learning system includes complex slave computing devices and a master computing device. Each slave computing device is configured to obtain a classification result using a weak classifier, and the master computing device is configured to receive the classification result from the slave computing devices and output a final classification result based on the classification result.

[0027] The main computing device and the slave computing device can be any processing device (also known as an edge device) located at the edge of the network and having computing capabilities, such as microcontrollers, personal computers (e.g., desktop computers or laptops), server computers, mobile devices (e.g., tablets or smartphones), wearable devices (e.g., smartwatches), network devices (e.g., routers), or in-vehicle information systems of autonomous vehicles, but this disclosure is not limited to these.

[0028] It should be noted that this disclosure does not limit the communication method between the master computing device and the slave computing device. They can communicate directly or indirectly in a wired manner, such as Inter-Integrated Circuit (I2C) or Ethernet, or they can communicate directly or indirectly in a wireless manner, such as Bluetooth or Wi-Fi. However, this disclosure does not limit them.

[0029] Generally, a weak classifier refers to a classification model that is small in scale but may have low classification accuracy. Because weak classifiers have low computational complexity and are quick to train, they are more suitable for edge devices with limited computing resources compared to other classification models. In one embodiment, a weak classifier can be a decision tree model, such as a single-layer decision stump or a decision tree with fewer layers. In another embodiment, a weak classifier can be a linear classifier, such as a perceptron.

[0030] It should be noted that this disclosure does not limit weak classifiers to the aforementioned smaller-scale classification models. Where computational resources allow, weak classifiers can be classification models implemented using k-Nearest Neighbors (k-NN), Support Vector Machines (SVM), Neural Networks (NN), etc., but this disclosure does not limit them.

[0031] It should be noted that this disclosure does not limit the classification algorithm used by each weak classifier. In other words, in the various embodiments of this disclosure, the classification algorithms used by the weak classifiers can all be the same, some can be the same, or all can be different. In one embodiment, the classification algorithm used by the weak classifier can be determined based on the computing power of the computing device. For example, the weak classifier of the computing device with poor computing power can be a single-layer decision tree, while the weak classifier of the computing device with better computing power can be a decision tree with more layers.

[0032] The training method for a weak classifier depends on the classification algorithm used. In one embodiment, the weak classifier uses a supervised machine learning algorithm (such as decision trees and support vector machines), which can learn using labeled initial sensing data. In another embodiment, the weak classifier uses an unsupervised machine learning algorithm (such as isolation forests), which can learn using unlabeled initial sensing data.

[0033] Each classification result can be a category identifier or a probability vector containing the probability of each category, but this disclosure is not limited to this. Taking anomaly classification as an example, the classification result can be a single value such as 0 or 1, where 0 can represent normal and 1 can represent abnormal. Alternatively, the classification result can be a probability vector of size 1x2, for example, [0.8, 0.2] can represent a probability of 0.8 being normal and a probability of 0.2 being abnormal.

[0034] The presentation format of the final classification result depends on the classification result. For example, if the classification result is a category identifier, the final classification result will also be presented as a category identifier. If the classification result is a category probability, the final classification result can be presented as a category probability, or further converted into a presentation as a category identifier, but this disclosure is not limited to this.

[0035] Figure 1 is a system architecture diagram of an integrated learning system 10 according to an embodiment of the present invention. As shown in Figure 1, the integrated learning system 10 includes a main computing device 11, slave computing devices 121, 122-12N, and sensors 131, 132-13N. The slave computing devices 121, 122-12N can communicate with the main computing device 11 and the individual corresponding sensors 131, 132-13N.

[0036] The main computing device 11 and the slave computing devices 121-12N can be any processing device (also known as an edge device) located at the edge of the network and having computing capabilities, such as microcontrollers, personal computers (e.g., desktop computers or laptops), server computers, mobile devices (e.g., tablets or smartphones), wearable devices (e.g., smartwatches), network devices (e.g., routers), or autonomous vehicles, but this disclosure does not limit them.

[0037] Sensors 131-13N can be any device used to detect events or changes in the environment and transmit information to other electronic devices, such as temperature and humidity sensors, accelerometers, ultrasonic sensors, sound sensors, image sensors, etc., but this disclosure is not limited to these. It should be noted that this disclosure does not limit the individual types of sensors 131-13N. In other words, in the embodiments of this disclosure, the types of sensors 131-13N can all be the same, some can be the same, or all can be different.

[0038] It should be noted that this disclosure does not limit the communication method between the main computing device 11, the slave computing devices 121-12N and the sensors 131-13N. They can communicate directly or indirectly in a wired manner, such as through an Inter-Integrated Circuit (I2C) or Ethernet, or they can communicate directly or indirectly in a wireless manner, such as through Bluetooth or Wi-Fi. However, this disclosure does not limit them.

[0039] Figure 2A is a data flow diagram of training phase 21 of an ensemble learning method according to an embodiment of the present invention. The ensemble learning system 10 executes this ensemble learning method, and training phase 21 is performed by the slave computing devices 121-12N therein. As shown in Figure 2A, training phase 21 includes steps S211 and S212. Steps S211 and S212 are described below.

[0040] In step S211, the computing devices 121 to 12N receive first sensing data 231 to 23N from the corresponding sensors 131 to 13N respectively.

[0041] The first sensing data 231~23N corresponds to the type of sensor 131~13N, which can be temperature and humidity values, light intensity values, acceleration values, vibration frequency values, sound intensity values ​​or coordinate values, etc., but this disclosure does not limit this.

[0042] In step S212, each of the weak classifiers 221 to 22N is trained from the computing devices 121 to 12N using their respective first sensing data 231 to 23N.

[0043] In one embodiment, the weak classifiers 221-22N can be decision tree models, such as single-layer decision stumps or decision trees with fewer layers. In another embodiment, the weak classifiers 221-22N can be linear classifiers, such as perceptrons.

[0044] It should be noted that this disclosure does not limit the weak classifiers 221-22N to the aforementioned small-scale classification models. Where the computational resources of the computing devices 121-12N allow, the weak classifiers 221-22N can be classification models implemented using k-Nearest Neighbors (k-NN), Support Vector Machine (SVM), Neural Network (NN), etc., but this disclosure does not limit them.

[0045] It should be noted that this disclosure does not limit the classification algorithms individually used by the weak classifiers 221-22N. In other words, in the embodiments of this disclosure, the classification algorithms used by the weak classifiers 221-22N can all be the same, some can be the same, or all can be different. In one embodiment, the classification algorithm used by the weak classifiers 221-22N can be determined based on the computing power of the computing devices 121-12N. For example, the weak classifier of the computing device 121-12N with weaker computing power can be a single-layer decision tree, while the weak classifier of the computing device 121-12N with stronger computing power can be a decision tree with more layers.

[0046] The training method for weak classifiers 221~22N depends on the classification algorithm used. In one embodiment, the weak classifier uses a supervised machine learning algorithm (such as decision trees and support vector machines), which can learn using labeled first-sensory data. In another embodiment, the weak classifier uses an unsupervised machine learning algorithm (such as isolation forests), which can learn using unlabeled first-sensory data.

[0047] In one embodiment, the first sensing data 231-23N have the same data format. For example, the first sensing data 231-23N may have the same data fields, such as temperature values. In this embodiment, the slave computing devices 121-12N can communicate with each other. Furthermore, each slave computing device 121-12N shares a portion of its individual first sensing data 231-23N with other slave computing devices, and receives the shared portions from other slave computing devices, and uses its respective first sensing data and shared portions to train its respective weak classifiers 221-22N.

[0048] It should be noted that this disclosure does not limit the communication method between the computing devices 121 to 12N. They can communicate directly or indirectly in a wired manner, such as through an Inter-Integrated Circuit (I2C) or Ethernet, or they can communicate directly or indirectly in a wireless manner, such as through Bluetooth or Wi-Fi. However, this disclosure does not limit them.

[0049] It should be noted that this disclosure does not limit the percentage of the first sensing data to be shared. In one embodiment, a slave computing device may share a fixed percentage (e.g., 20%) of the first sensing data with other slave computing devices. In another embodiment, the percentage of the shared data can be determined based on the computing power of the other slave computing devices. For example, for a slave computing device with better computing power, 90% of the first sensing data may be shared, while for a slave computing device with poorer computing power, only 5% of the first sensing data may be shared. However, this disclosure is not limited to this.

[0050] It is worth noting that by sharing initial sensor data with other computing devices, the amount of training data for weak classifiers can be increased, without being limited by the data collection capabilities of a single sensor. This will help improve the generalization ability and classification accuracy of weak classifiers.

[0051] Figure 2B is a data flow diagram of the inference stage 22 of an ensemble learning method according to an embodiment of the present invention. The inference stage 22 is executed in the ensemble learning system 10. As shown in Figure 2B, the inference stage 22 includes steps S221 to S224. Steps S221 to S224 are described below.

[0052] In step S221, the computing devices 121 to 12N receive second sensing data 241 to 24N from the corresponding sensors 131 to 13N respectively.

[0053] The second sensing data 241~24N also corresponds to the type of sensors 131~13N, which may be, for example, temperature and humidity values, light intensity values, acceleration values, vibration frequency values, sound intensity values, and coordinate values, but this disclosure is not limited to these.

[0054] In step S222, the classification results 251 to 25N of the second sensing data 241 to 24N are obtained from the computing devices 121 to 12N using their respective weak classifiers 221 to 22N.

[0055] The classification results 251~25N can each be a category identifier or a probability vector containing the probability of each category, but this disclosure does not limit this. Taking anomaly classification as an example, the classification results 251~25N can be a single value such as 0 or 1, where 0 can represent normal and 1 can represent abnormal. Alternatively, the classification results 251~25N can be a probability vector of size 1x2, for example, [0.8, 0.2] can represent a probability of 0.8 of being normal and a probability of 0.2 of being abnormal.

[0056] In step S223, the main processing unit 11 receives classification results 251 to 25N from the processing units 121 to 12N. In step S224, the main processing unit 11 outputs the final classification result 26 based on the classification results 251 to 25N.

[0057] In one embodiment, the main processing unit 11 takes the mode of the classification results 251~25N as the final classification result 26. In another embodiment, the main processing unit 11 takes a weighted average of the classification results 251~25N to obtain the final classification result 26. In other embodiments, the main processing unit 11 takes the maximum value of the classification results 251~25N as the final classification result 26, but this disclosure is not limited to this.

[0058] The presentation format of the final classification result 26 depends on the classification results 251 to 25N. For example, if the classification results 251 to 25N are category identifiers, the final classification result 26 will also be presented as a category identifier. If the classification results 251 to 25N are category probabilities, the final classification result 26 can be presented as a category probability, or further converted into a presentation as a category identifier, but this disclosure does not limit this.

[0059] It should be noted that in the above embodiments, even if the types of sensors 131-13N are different, the weak classifiers 221-22N should have a consistent classification target, such as judging normal / abnormal. Taking a building access control system as an example, the weak classifiers 221-22N in the computing devices 121-12N can determine whether the building has been intruded upon by using sensor data from sensors such as cameras, vibration sensors, and infrared sensors. Then, the main computing device 11 combines the above judgment results to obtain the final judgment result. By considering the judgment results of different weak classifiers from multiple perspectives, the problem of misjudgment by a single weak classifier can be effectively avoided, thereby improving the accuracy of the overall system judgment.

[0060] By combining the secondary computing devices 121-12N and the main computing device 11, the ensemble learning system 10 can form a strong classifier at the functional level. In embodiments where the weak classifiers 221-22N are all decision tree models, the ensemble learning system 10 forms a random forest model at the functional level.

[0061] It is worth noting that strong classifiers (such as random forests) typically have high model complexity and are not easily deployed on edge devices with limited computing resources. However, the ensemble learning system 10 can effectively utilize the computing resources of multiple edge devices to implement strong classifiers, such as random forest models, by distributing weak classifiers (such as decision trees in random forests) across different edge devices.

[0062] The embodiments disclosed herein provide another form of ensemble learning system. This ensemble learning system includes a complex number of slave computing devices, each slave computing device configured to share a weak classifier with other slave computing devices and receive other weak classifiers from other slave computing devices; and to obtain a final classification result using the weak classifiers and other weak classifiers.

[0063] The computing device, as described above, can be any processing device (also known as an edge device) located at the edge of the network and possessing computing capabilities. Communication between computing devices can be direct or indirect, via wired or wireless means.

[0064] Furthermore, the classification algorithms used by individual weak classifiers can vary depending on the computing resources of the processing device; they can all be the same, partially the same, or all different. The training method for weak classifiers will also depend on whether the classification algorithm used is a supervised or unsupervised machine learning algorithm.

[0065] Figure 3 is a system architecture diagram of an integrated learning system 30 according to one embodiment of the present invention. As shown in Figure 3, the integrated learning system 30 includes slave computing devices 321, 322-32N and sensors 331, 332-33N. The slave computing devices 321-32N can communicate with each other and can also communicate with individual corresponding sensors 331-33N.

[0066] The computing devices 321 to 32N can be any processing device (also known as an edge device) located at the edge of the network and possessing computing capabilities, such as microcontrollers, personal computers (e.g., desktop computers or laptops), server computers, mobile devices (e.g., tablets or smartphones), wearable devices (e.g., smartwatches), network devices (e.g., routers), or autonomous vehicles, but this disclosure does not limit them.

[0067] The types of sensors 331 to 33N must be consistent. They can all be one of the following: temperature and humidity sensor, acceleration sensor, ultrasonic sensor, sound sensor, image sensor, but this disclosure does not limit them.

[0068] It should be noted that this disclosure does not limit the communication methods between the computing devices 321-32N and between each other and the sensors 331-33N. They can communicate directly or indirectly in a wired manner, such as through an Inter-Integrated Circuit (I2C) or Ethernet, or they can communicate directly or indirectly in a wireless manner, such as through Bluetooth or Wi-Fi. However, this disclosure does not limit them.

[0069] Figure 4A is a data flow diagram of the training phase 41 of an ensemble learning method 40 according to an embodiment of the present invention. The training phase 41 is executed in the ensemble learning system 30. As shown in Figure 4A, the training phase 41 includes steps S411 to S413.

[0070] Accordingly, Figure 4B is a schematic diagram of the processing units 321-32N completing the training phase in this embodiment. Steps S411-S413 will be described below with reference to Figures 4A and 4B.

[0071] In step S411, the computing devices 321 to 32N receive first sensing data 431 to 43N from the corresponding sensors 331 to 33N respectively.

[0072] The type of the first sensing data 431-43N will depend on the sensors 331-33N, and may include temperature and humidity values, light intensity values, acceleration values, vibration frequency values, sound intensity values, and coordinate values, etc., but this disclosure is not limited to this. However, the data format of the first sensing data 431-43N must be consistent. For example, the first sensing data 231-23N must have the same data fields, such as temperature values.

[0073] In step S412, each of the weak classifiers 421 to 42N is trained from the computing devices 321 to 32N using their respective first sensing data 431 to 43N.

[0074] When the computational resources of computing devices 321-32N are limited, the weak classifiers 421-42N can be decision tree models, such as single-layer decision trees or decision trees with fewer layers, or linear classifiers such as perceptrons. Where the computational resources of computing devices 321-32N allow, the weak classifiers 421-42N can be classification models implemented using K-nearest neighbors, support vector machines, neural networks, etc., but this disclosure is not limited to these.

[0075] It should be noted that this disclosure does not limit the classification algorithms individually used by the weak classifiers 421-42N. In other words, the classification algorithms used by the weak classifiers 421-42N can all be the same, some can be the same, or all can be different; however, this disclosure does not limit this. In one embodiment, the classification algorithm used by the weak classifiers 421-42N can be determined based on the computing power of the computing devices 321-32N. For example, the weak classifier of the computing device 321-32N with weaker computing power can be a single-layer decision tree, while the weak classifier of the computing device 321-32N with stronger computing power can be a decision tree with more layers.

[0076] The training method for weak classifiers 421~42N will depend on the classification algorithm used. In one embodiment, the weak classifier uses a supervised machine learning algorithm (such as decision trees and support vector machines), which can learn using labeled first-sensory data. In another embodiment, the weak classifier uses an unsupervised machine learning algorithm (such as isolation forests), which can learn using unlabeled first-sensory data.

[0077] In one embodiment, each of the computing devices 321-32N shares a portion of its individual first sensing data 431-43N with other computing devices and receives the shared portions from other computing devices, and uses its respective first sensing data and shared portions to train its respective weak classifiers 421-42N.

[0078] It should be noted that this disclosure does not limit the percentage of the first sensing data to be shared. In one embodiment, a slave computing device may share a fixed percentage (e.g., 20%) of the first sensing data with other slave computing devices. In another embodiment, the percentage of the shared data can be determined based on the computing power of the other slave computing devices. For example, for a slave computing device with better computing power, 90% of the first sensing data may be shared, while for a slave computing device with poorer computing power, only 5% of the first sensing data may be shared. However, this disclosure is not limited to this.

[0079] It is worth noting that by sharing initial sensor data with other computing devices, the amount of training data for weak classifiers can be increased, without being limited by the data collection capabilities of a single sensor. This will help improve the generalization ability and classification accuracy of weak classifiers.

[0080] In step S413, the slave computing devices 321-32N share their respective weak classifiers 421-42N with other slave computing devices and receive other weak classifiers from other slave computing devices. Afterwards, as shown in Figure 4B, each slave computing device will contain trained weak classifiers 421-42N.

[0081] In one embodiment, the ensemble learning system 30 further includes a main computing device. In one embodiment, during the training phase 41, the main computing device evaluates whether the performance of weak classifiers 421-42N exceeds a specified threshold. The performance of weak classifiers 421-42N can be evaluated using common model performance metrics such as accuracy or precision, but this disclosure is not limited to this. Furthermore, during the training phase 41, in response to a weak classifier's performance not exceeding the specified threshold, the main computing device further prohibits this slave computing device from sharing its weak classifier with other slave computing devices. In one embodiment, during the training phase 41, the main computing device further determines whether to allow a slave computing device to receive other classifiers from other slave computing devices based on the slave computing device's current computing resources. For example, if a slave computing device's memory size is only tens of KB, it is not allowed to receive other classifiers from other slave computing devices.

[0082] It is worth noting that by controlling the transmission or reception of classifiers through the main computing unit, the number of weak classifiers included in the computing unit can be dynamically controlled. In other words, the main computing unit can be used to control the size of the strong classifiers in the computing unit, thereby achieving optimal utilization of computing resources.

[0083] Figure 4C is a data flow diagram of the inference stage 42 of the ensemble learning method according to an embodiment of the present invention. The inference stage 42 is executed in the ensemble learning system 30. As shown in Figure 4C, the inference stage 42 includes steps S421 and S422. Steps S421 and S422 are described below.

[0084] In step S421, the computing devices 321 to 332N receive second sensing data 441 to 44N from the corresponding sensors 331 to 33N respectively.

[0085] The second sensing data 441~44N also corresponds to the type of sensor 331~33N, which can be temperature and humidity values, light intensity values, acceleration values, vibration frequency values, sound intensity values, and coordinate values, etc., but this disclosure does not limit this.

[0086] In step S422, the computing devices 321-32N use weak classifiers 421-42N to classify their respective second sense data 441-44N to obtain their respective final classification results 451-45N. For example, the computing device 321 uses a strong classifier formed by weak classifier 421 and other weak classifiers 422-42N to classify the second sense data 441 to obtain the final classification result 451.

[0087] The final classification results 451~45N can be category identifiers or probability vectors containing the probabilities of each category, but this disclosure is not limited to either. Taking anomaly classification as an example, the classification results 451~45N can be a single value such as 0 or 1, where 0 represents normal and 1 represents anomalous. Alternatively, the classification results 451~45N can be a probability vector of size 1x2, for example, [0.8, 0.2] represents a probability of 0.8 of being normal and a probability of 0.2 of being anomalous.

[0088] It should be noted that although steps S421 and S422 are executed by all arithmetic devices 321-32N in Figure 4C, this disclosure does not limit the arithmetic devices 321-32N to executing steps S421 and S422 together. In other words, arithmetic devices 321-32N can execute steps S421 and S422 independently during the inference phase.

[0089] Figure 5 is a schematic diagram illustrating step S422 executed from the computing device 321 in Figure 4C according to an embodiment of the present invention. As shown in Figure 5, step S422 includes steps S501 and S502. Steps S501 and S502 will be described below.

[0090] In step S501, the second sensing data 441 is input from the computing device 321 to the weak classifiers 421-42N respectively to obtain individual classification results 521-52N. In step S502, the computing device 321 outputs the final classification result 451 based on the classification results 521-52N.

[0091] In one embodiment, the processing unit 321 takes the mode of classification results 521~52N as the final classification result 451. In another embodiment, the processing unit 321 takes a weighted average of classification results 251~25N to obtain the final classification result 451. In other embodiments, the processing unit 321 takes the maximum value of classification results 521~52N as the final classification result 451, but this disclosure is not limited to this.

[0092] The classification results 521-52N can be category identifiers or probabilities for each category, but this disclosure is not limited to this. In the above embodiments, the presentation format of the classification results 521-52N will affect the final classification result 451. For example, when the classification results 521-52N are category identifiers, the final classification result 26 will also be presented as a category identifier. When the classification results 251-25N are category probabilities, the final classification result 26 will also be presented as a category probability, or further converted to a category identifier format, but this disclosure is not limited to this.

[0093] It is worth noting that for each computing device, considering the classification results of different weak classifiers from multiple perspectives can effectively avoid the problem of classification errors by a single weak classifier, thereby improving the classification accuracy of the computing device.

[0094] Furthermore, by sharing weak classifiers, a strong classifier can be formed from within the computing device 321. In an embodiment where all weak classifiers 421-42N are decision tree models, the weak classifiers 421-42N form a random forest model from within the computing device 321.

[0095] It is worth noting that strong classifiers (such as random forests) typically have high model complexity, making them difficult to train on edge devices with limited computing resources. However, the ensemble learning system 30 effectively utilizes the computing resources of multiple edge devices to train strong classifiers (such as random forests) by distributing weak classifiers (such as decision trees in random forests) across different edge devices. The trained strong classifier is then deployed on a single edge device.

[0096] The ensemble learning system and method disclosed herein enable ensemble learning training and inference on edge computing systems. Furthermore, by training weak classifiers on individual edge devices and deploying the strong classifier, formed by merging these weak classifiers, across the entire edge computing system or on individual edge devices, the accuracy of edge-side inference can be improved, while the latency caused by data transmission to the cloud can be reduced, thereby shortening model training time and enhancing real-time inference capabilities.

[0097] The preceding paragraphs describe various forms. Clearly, the teachings herein can be implemented in multiple ways, and any particular architecture or functionality disclosed in the examples is merely representative. Based on the teachings herein, it should be understood that the individual forms disclosed herein can be implemented independently, or two or more forms can be implemented in combination.

[0098] Although the present disclosure has been described above with reference to embodiments, it is not intended to limit the present disclosure. Anyone skilled in the art may make some modifications and refinements without departing from the spirit and scope of the present disclosure. Therefore, the scope of protection of the invention shall be determined by the claims attached.

[0099] 10: Integrated Learning System 11: Main Computing Unit 121, 122~12N: From the arithmetic unit 131, 132~13N: Sensors 21: Training Phase S211~S212: Steps 221, 222~22N: Weak classifiers 231, 232~23N: First sensing data 22: Inference Stage S221~S224: Steps 241, 242~24N: Second sensing data 251, 252~25N: Classification results 26: Final classification results 30: Integrated Learning System 321, 322~32N: From the arithmetic unit 331, 332~33N: Sensors 41: Training Phase S411~S412: Steps 421, 422~42N: Weak classifiers 431, 432~43N: First sensing data 42: Inference Stage S421~S422: Steps 441, 442~44N: Second sensing data 451, 452~45N: Final classification results S501~S502: Steps 521, 522~52N: Classification results

Claims

1. An ensemble learning system, comprising: A plurality of slave computing devices, wherein each slave computing device is configured to: in a training phase, receive first sensing data from a sensor and train a weak classifier using the first sensing data; in an inference phase, receive second sensing data from the sensor and obtain a classification result of the second sensing data using the weak classifier; and a master computing device is configured to: in the inference phase, receive the classification results from the slave computing devices; and output a final classification result based on the classification results.

2. The integrated learning system as described in claim 1, wherein the main computing device is further configured to: during the inference phase, take the mode of the classification results as the final classification result.

3. The ensemble learning system as described in claim 1, wherein the weak classifier of each slave computing device is a decision tree model.

4. An integrated learning system comprising complex slave computing devices, wherein each slave computing device is configured to: in a training phase: receive first sensing data from a sensor; train the weak classifier using the first sensing data; share the weak classifier with other slave computing devices and receive other weak classifiers from the other slave computing devices; and in an inference phase: receive second sensing data from the sensor; and classify the second sensing data using the weak classifier and the other weak classifiers to obtain a final classification result.

5. The integrated learning system as described in claim 4, wherein the first sensing data of each slave computing device has the same data format.

6. The integrated learning system as described in claim 5, wherein each slave computing device is further configured to perform the following operations during the inference phase: inputting the second sensed data to the weak classifier and the other weak classifiers respectively to obtain individual classification results; and outputting the final classification result based on the classification results.

7. The integrated learning system as described in claim 6, wherein each slave computing device is further configured to: during the inference phase, take the mode of the classification results as the final classification result.

8. The ensemble learning system as described in claim 5, wherein the weak classifier of each slave computing device is a decision tree model.

9. An ensemble learning method, executed in a computer system including a complex number sub-processing unit and a main processing unit, the ensemble learning method comprising: In a training phase: Each slave computing device receives first sensing data from a sensor and uses the first sensing data to train the weak classifier. And in an inference phase: receiving second sensing data from the sensor through each slave computing device, and obtaining the classification result of the second sensing data using the weak classifier; and receiving the classification results from the slave computing devices through the master computing device, and outputting a final classification result based on the classification results.

10. The ensemble learning method as described in claim 9 further includes, during the training phase, performing the following operations through each slave computing device: sharing a portion of the first sensing data to other slave computing devices, and receiving the shared portions from the other slave computing devices; and training the weak classifier using the first sensing data and the shared portions; wherein, The data format of the first sensing data from each computing device is the same.

11. An ensemble learning method, implemented in a computer system including a complex number of slave computing devices, the ensemble learning method comprising performing the following operations through each slave computing device: In a training phase: receiving first sensing data from a sensor; training a weak classifier using the first sensing data; sharing a weak classifier with other slave computing devices and receiving other weak classifiers from the other slave computing devices; and in an inference phase: receiving second sensing data from the sensor; and classifying the second sensing data using the weak classifier and the other weak classifiers to obtain a final classification result.

12. The integrated learning method as described in claim 11, wherein the data format of the first sensing data from each slave computing device is the same.

13. The ensemble learning method as described in claim 12 further includes performing the following operations during the training phase by each slave computing device: sharing a portion of the first sensing data to other slave computing devices and receiving the shared portions from the other slave computing devices; and training the weak classifier using the first sensing data and the shared portions.

14. The ensemble learning method as described in claim 12, wherein the computer system further includes a main computing device, and the ensemble learning method further includes performing the following operations on each slave computing device during the training phase via the main computing device: evaluating whether the performance of the weak classifier exceeds a specified threshold; and disallowing the slave computing device from sharing the weak classifier to other slave computing devices in response to the weak classifier's performance not exceeding the specified threshold.

Images