Classification method and electronic device

Abstract

The present disclosure provides a classification method and an electronic device. The classification method includes the following steps. First feature data of a plurality of assembly pictures is extracted, wherein each assembly picture contains an operator at a station. The first feature data is converted into a first feature vector. Second feature data of personal data of the operator is converted into a second feature vector. The first feature vector and the second feature vector are merged into a first feature matrix. The first feature matrix is classified to obtain a classification result corresponding to the efficiency of the operator operating at the station. Accordingly, the present disclosure can effectively predict the work efficiency of each operator at each station.

Description

Technical Field

[0001] This disclosure relates to a classification method and apparatus, and more particularly to a classification method and electronic device for personnel allocation. Background Technology

[0002] In production line management, personnel placement impacts production capacity. Generally, personnel allocation methods rely on historical data and are designed with allocation logic tailored to business characteristics, but these methods cannot be automatically updated. Therefore, changes in business operations or temporal and spatial context can render these methods ineffective, affecting production capacity. Furthermore, past personnel allocation methods built using machine learning models require correctly labeled data for analysis and training, making them susceptible to limitations in sample size. Moreover, training machine learning models relies on historical data to create effective predictive features for accurate predictions. Consequently, past personnel allocation methods over-rely on structured data, leading to inaccurate predictions or low feature relevance. In terms of method expansion, applying traditional machine learning to other scenarios requires retraining a different machine learning model. Therefore, even slight changes in the projects requiring prediction necessitate redesigning features and algorithms, hindering the efficient and rapid integration of machine learning models. Summary of the Invention

[0003] This disclosure provides a classification method and electronic device that can effectively predict the work efficiency of each operator at each station.

[0004] The classification method disclosed herein includes: extracting first feature data from multiple assembled images, wherein each assembled image contains an operator located at a station; converting the first feature data into a first feature vector; converting second feature data containing the operator's personal data into a second feature vector; merging the first feature vector and the second feature vector into a first feature matrix; and classifying the first feature matrix to obtain a classification result corresponding to the efficiency of the operator's operation at the station.

[0005] In the embodiments disclosed herein, before the step of extracting the first feature data of the assembly image, the method further includes: recording an image stream of the operator performing the assembly operation at the station; and acquiring multiple consecutive image frames from the image stream as the assembly image.

[0006] In the embodiments disclosed herein, the step of extracting the first feature data corresponding to the assembled image includes: extracting the first feature data corresponding to the assembled image using a feature extractor, wherein the feature extractor includes a feature detector of an autoencoder or a convolutional neural network.

[0007] In the embodiments disclosed herein, after classifying the first feature matrix, the classification method further includes: obtaining multiple first scores for the operator corresponding to multiple efficiency tags; and taking the one of the efficiency tags corresponding to the highest first score as the classification result.

[0008] In the embodiments disclosed herein, the classification method further includes: extracting third feature data of the assembled image, wherein the first feature data is different from the third feature data; converting the third feature data into a third feature vector; merging the third feature vector and the second feature vector into a second feature matrix; classifying the second feature matrix to obtain multiple second scores corresponding to the efficiency label of the operator; and obtaining the classification result corresponding to the efficiency of the operator's operation at the station based on the first score and the second score.

[0009] In the embodiments disclosed herein, the step of converting the first feature data into the first feature vector includes: converting the first feature data into the first feature vector using a flattening function, a fully connected function, and an activation function.

[0010] In the disclosed embodiments, the step of classifying the first feature matrix includes: classifying the first feature matrix using a classifier. The classification method further includes: using a clustering algorithm to cluster multiple yield data points from multiple operators at the station corresponding to different dates, to obtain an efficiency label for each operator at each time point; and using the efficiency label for each operator at each time point, and multiple historical data points for each operator to train the classifier, wherein each historical data point includes the operator's personal data and historical images taken during assembly operations at the station.

[0011] In the embodiments disclosed herein, the personal data includes gender, seniority, accommodation, vision, production line position, record date, age, assembly yield, or any combination thereof.

[0012] In the embodiments disclosed herein, before the step of converting the second feature data containing the operator's personal data into the second feature vector, the method further includes: using a time series module to predict changes in the personal data within a specified time range, and using the predicted personal data as the second feature data.

[0013] The disclosed electronic device for classifying operators includes a memory and a processor. The memory stores at least one code segment, multiple assembly images, and personal data of the operators. The processor is coupled to the memory and configured to execute the at least one code segment to implement the classification method.

[0014] Based on the above, this disclosure can solve the dilemma that it is difficult to find effective features with only structured data in the past, and improve the accuracy of the final result by making predictions based on multiple data. Attached Figure Description

[0015] Figure 1 This is a block diagram of an electronic device according to an embodiment of the present disclosure;

[0016] Figure 2 This is an architectural diagram of a classification module according to an embodiment of the present disclosure;

[0017] Figure 3 This is a flowchart of a classification method according to an embodiment of the present disclosure;

[0018] Figure 4 This is an architectural diagram of a classification module according to an embodiment of the present disclosure;

[0019] Figure 5 This is an architectural diagram of a classification module according to an embodiment of the present disclosure.

[0020] Explanation of reference numerals in the attached figures

[0021] 100: Electronic devices

[0022] 110: Processor

[0023] 120: Memory

[0024] 121: Classification Module

[0025] 122: Database

[0026] 210: Feature Extractor

[0027] 220: First Feature Conversion Module

[0028] 230: Second Feature Conversion Module

[0029] 240: First Merging Module

[0030] 250: First Classifier

[0031] 41, 42: Analysis Module

[0032] 410: Automatic Encoder

[0033] 420: Feature Detector

[0034] 421: Third Feature Conversion Module

[0035] 423: Second Merging Module

[0036] 425: Second Classifier

[0037] 440: Classification Results

[0038] 510: Timing Module

[0039] 520: Personnel Allocation Module

[0040] D: Personal Data

[0041] F1: First Feature Data

[0042] F2: Second Feature Data

[0043] F3: Third Feature Data

[0044] M: Multiple assembly images

[0045] S305~S325: Steps in the classification method Detailed Implementation

[0046] Figure 1 This is a block diagram of an electronic device according to an embodiment of the present disclosure. Please refer to... Figure 1 The electronic device 100 includes a processor 110 and a memory 120. The processor 110 is coupled to the memory 120.

[0047] The processor 110 may be, for example, a central processing unit (CPU), a physical processing unit (PPU), a programmable microprocessor, an embedded control chip, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or other similar devices.

[0048] The memory 120 may be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk, or other similar device or combination thereof. The memory 120 stores multiple code segments that, upon being loaded, will be executed by the processor 110. For example, the memory 120 includes a classification module 121. The classification module 121 consists of one or more code segments executed by the processor 110 to implement a classification method.

[0049] The memory 120 also includes a database 122. Database 122 stores multiple yield data points and historical data points for multiple operators at the same or different workstations on different dates. This historical data includes individual data for each operator and historical images taken during assembly operations at the workstations. The processor 110 first groups the multiple operators based on the yield data stored in database 122 and assigns each operator an efficiency label (e.g., good, average, poor) at different time points. Then, it uses the efficiency labels for each operator on different dates and the historical data of these operators to train the classification module 121, which is then used for prediction.

[0050] For example, processor 110 can use the K-means algorithm to group operators into multiple groups based on multiple yield data points, with each group corresponding to an efficiency label. Here, assuming there are three efficiency labels: "Good," "Medium," and "Poor," the K value of the K-means algorithm is set to 3 for grouping. This defines a group with a high defect rate as "Poor," a group with a medium defect rate as "Medium," and a group with a low defect rate as "Good." This method marks the efficiency label for each operator at a specific station each day, as shown in Table 1. Table 1 records each operator's individual data and their efficiency label at a specific station on a specific day.

[0051] Table 1

[0052]

[0053] After the efficiency labels for each operator are completed, the historical data corresponding to each operator can be used to train the classification module 121. For example, Figure 2 This is an architectural diagram of a classification module according to an embodiment of the present disclosure. In this embodiment, the classification module 121 employs a feature acquirer 210, which acquires features from multiple assembly images M and integrates them into first feature data F1. Here, these assembly images M are historical images and are stored in the memory 120. The multiple assembly images are acquired by recording an image stream of operator A performing assembly work at the station, and then acquiring multiple consecutive image frames from the image stream as the multiple assembly images M. In some other embodiments, the multiple assembly images M may also be images taken by operator A at different time points during assembly work at the station (e.g., before assembly, during assembly, and after assembly). The purpose of acquiring features from the multiple assembly images M and integrating them into the first feature data F1 is that these assembly images M can represent some assembly habits, assembly postures, assembly order, or assembly efficiency of operator A during assembly.

[0054] Here, it is assumed that multiple assembly images M and personal data D of an operator A, who has been labeled with efficiency tags as recorded in Table 1, are used as input data. The assembly images M and personal data D are stored in memory 120. Personal data D includes gender, seniority, accommodation, vision, production line position, record date, age, and assembly yield, etc. The personal data D is digitized to obtain second feature data F2. After obtaining first feature data F1 and second feature data F2, first feature conversion module 220 and second feature conversion module 230 are used to perform feature conversion on first feature data F1 and second feature data F2 respectively. Then, first merging module 240 is used to merge them, and the merged result is input into first classifier 250 to obtain scores corresponding to multiple efficiency tags. The efficiency tag corresponding to the highest score is used as the final classification result. Then, the final classification result is compared with the pre-labeled efficiency tags to adjust the parameters and / or weights in the first classifier 250 of classification module 121.

[0055] Figure 3 This is a flowchart of a classification method according to an embodiment of this disclosure. Please refer to... Figure 3 In step S305, the first feature data F1 of the multiple assembled images M is extracted. Here, the feature extractor 210 obtains the first feature data F1 from the multiple assembled images M. The feature extractor 210 can be implemented using a trained autoencoder. For example, the autoencoder is trained by using multiple assembly images of each operator as input to reconstruct the image through deep learning, and the features of the multiple assembly images used to reconstruct the input are extracted as the first feature data. Alternatively, the feature extractor 210 can also include a feature detector in a Convolutional Neural Network (CNN). Generally, a feature map can be obtained in the convolutional layers of a CNN using a feature detector (also called a kernel or filter).

[0056] Next, in step S310, the first feature data F1 is transformed into a first feature vector. For example, the first feature transformation module 220 includes a flattening function, a fully connected (FC) function, and an activation function. First, the flattening function is used to flatten the first feature data F1. That is, the multidimensional first feature data F1 is transformed into a one-dimensional matrix. Then, the one-dimensional matrix is ​​input to the fully connected function, and a feature vector is obtained by adjusting the weights and biases. After that, the feature vector is input to the activation function to obtain the first feature vector.

[0057] Here, "fully connected function" refers to the processing method of a fully connected layer in a neural network. A fully connected function can be, for example, a matrix multiplication, equivalent to a feature space transformation. For instance, a fully connected function can be used to perform an affine transformation on the input data, linearly transforming one vector space to another and extracting and integrating useful information. The activation function can be a rectified linear unit (ReLU) function, such as a ramp function, to enhance nonlinear characteristics.

[0058] In step S315, the second feature data F2, which records the operator's personal data D, is transformed into a second feature vector. For example, the second feature transformation module 230 includes a fully connected function and an activation function. The fully connected function linearly transforms the vector space of the second feature vector to another vector space, and then an activation function (e.g., the ReLU function) is used to enhance the nonlinear characteristics.

[0059] Next, in step S320, the first eigenvector and the second eigenvector are merged into a first feature matrix. Here, the first merging module 240 includes a concat function, a fully connected function, and an activation function. The concat function merges the first eigenvector and the second eigenvector into a matrix, then the fully connected function linearly transforms the vector space of the matrix to another vector space, and finally, an activation function (e.g., the ReLU function) is used to enhance the nonlinear characteristics. This yields the first feature matrix.

[0060] Next, in step S325, the first feature matrix is ​​classified to obtain a classification result corresponding to the efficiency of the operator's operation at the station. Specifically, the first feature matrix is ​​input to the first classifier 250 to obtain a classification result. The first classifier 250 obtains the scores of multiple efficiency labels for the operator based on the input first feature matrix. Here, the first classifier 250 can be implemented using a fully connected function. In this embodiment, multiple fully connected layers (fully connected functions) are used, with the last fully connected layer acting as the classifier, while the other fully connected layers are used for feature acquisition. Then, the processor 110 compares the classification result of the first classifier 250 with the pre-labeled efficiency labels, thereby adjusting the parameters and / or weights in the first classifier 250 of the classification module 121. After training, the first classifier 250 can be used to classify the data corresponding to unlabeled operators.

[0061] In another embodiment, the classification module 121 may also be implemented using two different feature extractors and two different classifiers, as illustrated below.

[0062] Figure 4 This is an architectural diagram of a classification module according to an embodiment of the present disclosure. In this embodiment, the classification module 121 employs two sets of analysis modules 41 and 42. Analysis module 41 includes an autoencoder 410, a first feature conversion module 220, a first merging module 240, and a first classifier 250. Analysis module 42 includes a feature detector 420, a third feature conversion module 421, a second merging module 423, and a second classifier 425. Other components with the same reference numerals represent the same functions. Here, the autoencoder 410 needs to be pre-trained before use, while the feature detector 420 adjusts its internal parameters along with the training process of the first classifier 250 of the classification module 121.

[0063] In analysis module 41, the autoencoder 410 obtains first feature data F1 from multiple assembly images M corresponding to operators labeled with efficiency tags. After obtaining the first feature data F1, the first feature data F1 is converted into a first feature vector by the first feature conversion module 220. Then, the first merging module 240 merges the first feature vector and the second feature vector to obtain a first feature matrix, and the merged first feature matrix is ​​input into the first classifier 250 to obtain a first score corresponding to multiple efficiency tags. That is, each efficiency tag has a corresponding first score.

[0064] On the other hand, in the analysis module 42, the feature detector 420 obtains third feature data F3 from multiple assembly images M corresponding to operators labeled with efficiency tags. After obtaining the third feature data F3, the third feature data F3 is converted into a third feature vector by the third feature conversion module 421. Here, the third feature conversion module 421 is similar to the first feature conversion module 220. Then, the second merging module 423 merges the third feature vector and the second feature vector to obtain a second feature matrix, and inputs the merged second feature matrix into the second classifier 425 to obtain a second score corresponding to multiple efficiency tags. That is, one efficiency tag has a corresponding second score.

[0065] Finally, the final classification result 440 is obtained based on the first score obtained by the first classifier 250 and the second score obtained by the second classifier 425. For example, the classification result 440 records a score of 0.7 for the efficiency label "good", a score of 0.2 for the efficiency label "medium", and a score of 0.1 for the efficiency label "poor".

[0066] For example, Table 2 shows the first and second scores corresponding to different efficiency labels, and their corresponding weights. The weights can be determined based on the importance of the feature extractor; the higher the importance, the greater the weight.

[0067] Table 2

[0068] Efficiency Tag First score Second score good A1 B1 medium A2 B2 Poor A3 B3 Weight W1 W2

[0069] For example, the overall score S with an efficiency label of "good" good =A1×W1+B1×W2, the overall score S with an efficiency label of "moderate" middle =A2×W1+B2×W2, the overall score S with the efficiency label "poor" bad =A3×W1+B3×W2. Then, the efficiency label corresponding to the highest comprehensive score is taken as the final classification result.

[0070] In other embodiments, three or more sets of feature extractors, feature transformation modules, merging modules, and classifiers may also be employed.

[0071] Figure 5 This is an architectural diagram of a classification module according to an embodiment of this disclosure. Please refer to... Figure 5 This embodiment and Figure 4 The architectures shown are largely the same, the difference being... Figure 5 The architecture shown also includes a time-series module 510 and a personnel allocation module 520. In this embodiment, a specified time range for predicting the work efficiency of operators can be further set, and the time-series module 510 is used to predict the changes in personal data D within the specified time range, and the second feature data F2 is extracted based on the predicted personal data D. For example, assuming the specified time range is the next 90 days, the time-series module 510 is used to predict the changes in personal data D for each day of the next 90 days, such as whether the operator is still employed, whether they continue to stay overnight, etc. Then, the classification module 121 is used to predict the classification result of the operator at each station for each day of the next 90 days (the efficiency label is a score of "good", "medium" and "poor"). Then, the personnel allocation module 520 is used to allocate operators to obtain the optimal personnel configuration. The personnel allocation module 520 uses a linear programming algorithm to allocate operators to the best stations.

[0072] For example, using classification module 121 to predict operator U 01 In the next 90 days, at the station Sta 01The classification results for each day are calculated, and an efficiency score is assigned based on the classification result. For example, the efficiency scores for the efficiency labels "Good," "Medium," and "Poor" are 0, 1, and 2, respectively. As shown in Table 3, assuming the classification result for day 1 is "Good," its efficiency score is 0; the classification result for day 2 is "Poor," its efficiency score is 2; the classification result for day 3 is "Medium," its efficiency score is 1, and so on, until the efficiency score corresponding to the classification result for day 90 is 1. Then, the 90 prediction results are summed to obtain a prediction score (Score). 01 U 01 ).

[0073] Table 3

[0074]

[0075] Similarly, the classification module 121 is used to calculate the predicted scores for multiple operators at multiple different stations. After obtaining the predicted score for each operator at each station, operators can be further configured according to the required number of personnel at these stations, so that the total predicted scores of these configured operators are the lowest possible configuration score.

[0076] For example, suppose the position is Sta 01 With position Sta 02 The combined requirement for both is 4 people, stationed at Sta. 03 With position Sta 04 The combined requirement is 2 people, stationed at Sta. 01 There are currently 2 positions available for assignment, position Sta. 02 There are currently 6 positions available for assignment, with the position being Sta. 03 There are currently 8 positions available for distribution, with the position being Sta. 04 There are currently 3 locations available for distribution.

[0077] Table 4 below shows which station each operator will be assigned to, where X1 to X... i The value is either 1 or 0. A value of 1 indicates that the operator has been assigned to the corresponding station; a value of 0 indicates that the operator has not been assigned to the corresponding station. For example, operator U... 01 To explain, when operator U 01 Assigned to position Sta 01 When X1 is 1, the values ​​of X2, X3, and X4 are all 0. That is, when the value of any one of X1, X2, X3, and X4 is 1, the values ​​of the other three are all 0.

[0078] Table 4

[0079]

[0080] Based on the currently available locations and the required number of personnel at each station, the following restrictions are set:

[0081] SUM 01 =X1 + X5 + X9 + ... + X i-7 +X i-3 ≤2;

[0082] SUM 02 =X2 + X6 + X 10 +...+X i-6 +X i-2 ≤6;

[0083] SUM 03 =X3 + X7 + X 11 +...+X i-5 +X i-1 ≤8;

[0084] SUM 04 =X4+X8+X 12 +...+X i-1 +X i ≤3;

[0085] SUM 01 +SUM 02 =4;

[0086] SUM 03 +SUM 04 =2.

[0087] SUM 01 To indicate the position Sta 01 Total number of users, SUM 02 To indicate the position Sta 02 Total number of users, SUM 03 To indicate the position Sta 03 Total number of users, SUM 04 To indicate the position Sta 04 Total number of personnel allocated. Based on each station's Sta 01 ~Sta 04 The currently available distribution location is set to SUM. 01 ≤2; SUM 02 ≤6; SUM 03 ≤8; SUM 04 ≤3. Furthermore, the number of positions is set based on the required number of people (Station Sta). 01 With position Sta 02 The combined requirement for both is 4 people, stationed at Sta. 03 With position Sta 04The combined requirement for both is 2 people, set as SUM. 01 +SUM 02 =4; SUM 03 +SUM 04 =2.

[0088] Under the aforementioned constraints, the optimal personnel allocation is determined by using the predicted score of each operator at each station. That is, the lower the final allocation score, the better the desired results.

[0089] Furthermore, if new operator data is added to new stations on the production line in the future, it is not necessary to retrain the entire model architecture using all the data. Only a small number of data points from each station need to be provided to update the training model, which can then predict the performance of each operator at the new station.

[0090] In summary, this method can group historical data by performance, assigning an efficiency label to each data point. A feature extractor extracts features from each operator's assembly images, integrating these features with historical data. This addresses the challenge of finding effective features from structured data alone. Furthermore, after training, the results of multiple models are integrated for prediction, distinguishing the performance of each operator. Finally, linear programming is used to prioritize assigning high-performing operators to suitable workstations.

Claims

1. A classification method, characterized in that, include: Extract first feature data from multiple assembled images, each of which contains an operator located at a station; Transform the first feature data into a first feature vector; The second feature data containing the operator's personal data is converted into a second feature vector; The first feature vector and the second feature vector are combined to form the first feature matrix; as well as The first feature matrix is ​​classified to obtain a classification result corresponding to the efficiency of the operator's operation at the station.

2. The classification method according to claim 1, characterized in that, Before the step of extracting the first feature data of the assembled image, the method further includes: Record an image stream of the operator performing assembly work at the aforementioned station; and Multiple consecutive image frames from the image stream are obtained as the assembled image.

3. The classification method according to claim 1, characterized in that, The step of extracting the first feature data corresponding to the assembled image includes: The first feature data corresponding to the assembled image is extracted using a feature extractor, wherein the feature extractor includes a feature detector of an autoencoder or a convolutional neural network.

4. The classification method according to claim 1, characterized in that, After classifying the first feature matrix, the classification method further includes: Obtain multiple first scores for the operator corresponding to multiple efficiency tags; and The one that corresponds to the highest first score among the efficiency labels is taken as the classification result.

5. The classification method according to claim 4, characterized in that, Also includes: Extract the third feature data of the assembled image, wherein the first feature data is different from the third feature data; Convert the third feature data into a third feature vector; The third feature vector and the second feature vector are combined to form the second feature matrix; The second feature matrix is ​​classified to obtain multiple second scores for the operator corresponding to the efficiency label; as well as Based on the first score and the second score, the classification result corresponding to the efficiency of the operator's operation at the station is obtained.

6. The classification method according to claim 1, characterized in that, The steps of converting the first feature data into the first feature vector include: The first feature data is transformed into the first feature vector using a flattening function, a fully connected function, and an activation function.

7. The classification method according to claim 1, characterized in that, The steps for classifying the first feature matrix include: The first feature matrix is ​​classified using a classifier; The classification method further includes: A clustering algorithm is used to cluster multiple yield data points for multiple operators at the specified workstations on different dates to obtain the efficiency label for each operator at each time point; and The classifier is trained using the efficiency label corresponding to each operator at each time point and multiple historical data of each operator, wherein each historical data includes the personal data of each operator and historical images taken when performing assembly operations at the station.

8. The classification method according to claim 1, characterized in that, The personal data includes gender, seniority, accommodation, vision, production line position, record date, age, assembly yield, or any combination thereof.

9. The classification method according to claim 1, characterized in that, Before the step of converting the second feature data containing the operator's personal data into the second feature vector, the method further includes: The time series module is used to predict changes in the personal data within a specified time range, and the predicted personal data is used as the second feature data.

10. An electronic device for classifying operators, characterized in that, include: The memory stores at least one code segment, multiple assembly images, and the operator's personal data; as well as A processor, coupled to the memory, is configured to execute the at least one code segment to achieve the following: Extract first feature data from the assembled images, wherein each assembled image contains the operator located at the station; Transform the first feature data into a first feature vector; The second feature data containing the aforementioned personal data is transformed into a second feature vector; The first eigenvector and the second eigenvector are combined to form a first eigenmatrix; and The first feature matrix is ​​classified to obtain a classification result corresponding to the efficiency of the operator's operation at the station.

Images