Activity recognition based on image and computer-readable media

Abstract

An activity recognition method and a computer-readable media are disclosed. The activity recognition method is applied to an activity recognition system configured to recognize a number of activities. The activity recognition method includes: obtaining an original activity image, which has a number of pixels one-to-one corresponding to a number of sensors, corresponding to a first sampling time; determining an image feature according to a second activity corresponding to a second sampling time prior to the first sampling time; integrating the original activity image and the image feature to generate an characteristic activity image; determining a first activity corresponding to the first sampling time according to the characteristic activity image.

Description

[Technical Field]

[0001] This invention relates to an image-based behavior recognition method and a computer-readable medium. [Previous Technology]

[0002] Smart living environments offer convenience and security for people living alone. One method of behavior recognition in smart living environments involves installing sensors throughout the residence to determine the resident's behavior based on the triggered sensors. However, some dissimilar behaviors may trigger similar sensors, making it difficult to distinguish between them. Therefore, improving the accuracy of these distinctions is currently a trend. [Summary of the Invention]

[0003] One aspect of the present invention discloses a behavior recognition method applied to a behavior recognition system for recognizing a plurality of behaviors. The behavior recognition method includes: acquiring an original behavior image corresponding to a first time, the original behavior image including a plurality of pixels corresponding to whether a plurality of sensors are triggered; determining an image feature based on a second behavior corresponding to a second time, the second time preceding the first time; integrating the original behavior image and the image feature to generate a behavior image with features; and determining a first behavior corresponding to the first time based on the behavior image with features.

[0004] Another aspect of the present invention discloses a computer-readable medium. When the computer-readable medium is executed by a processing unit of a behavior recognition system for identifying a plurality of behaviors, the processing unit performs the following: acquiring an original behavior image corresponding to a first time, the original behavior image including a plurality of pixels corresponding to whether a plurality of sensors are triggered; determining an image feature based on a second behavior corresponding to a second time, the second time preceding the first time; integrating the original behavior image and the image feature to generate a behavior image with features; and determining a first behavior corresponding to the first time based on the behavior image with features.

[0005] In order to better understand the above and other aspects of the present invention, specific embodiments are described below in conjunction with the accompanying drawings:

Implementation Method

[0007] Please refer to Figure 1, which illustrates a block diagram of a behavior recognition system according to an embodiment of the present invention. The behavior recognition system 10 includes a plurality of sensors 102-1 to 102-n and a computing module 104. The sensors 102-1 to 102-n may include temperature sensors, sound sensors, light sensors, infrared sensors, and / or pressure sensors, etc. These sensors 102-1 to 102-n can be installed in various locations in a residence. For example, an infrared sensor can be installed above the entrance hall to detect whether someone is passing by, a pressure sensor can be installed inside the door frame of the entrance hall to detect whether the entrance door has been opened or closed, and a pressure sensor can be installed on a sofa to detect whether someone is sitting down, etc. In one embodiment, the behavior recognition system 10 can recognize human behavior.

[0008] The computing module 104 includes a storage unit 1041 and a processing unit 1043. The storage unit 1041 may be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, phase change memory, hard disk drive (HDD), register, solid state drive (SSD), or similar elements or combinations thereof. The processing unit 1043 may be, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microcontroller (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), graphics processing unit (GPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar elements or combinations thereof.

[0009] In one embodiment, at multiple times, each sensor 102-1 to 102-n transmits a signal to the computing module 104 depending on whether it has been triggered at that time. The computing module 104 records which of the sensors 102-1 to 102-n have been triggered and / or which have not been triggered at each time to generate a sensing record corresponding to that time, wherein a sampling interval separates two adjacent times. The sensing records may be stored in the storage unit 1041 of the computing module 104. The processing unit 1043 of the computing module 104 generates a raw behavioral image based on each sensing record. For example, at a first time, the computing module 104 acquires a first sensing record through the sensors 102-1 to 102-n and generates a first raw behavioral image based on the first sensing record; at a second time, the computing module 104 acquires a second sensing record through the sensors 102-1 to 102-n and generates a second raw behavioral image based on the second sensing record.

[0010] The original behavior image can be stored in storage unit 1041. The original behavior image can be a bitmap image, including multiple pixels corresponding to sensors 102-1 to 102-n. For example, the number of sensors 102-1 to 102-n is fifty, and the size of the original behavior image can be ten pixels long and ten pixels wide. Among the one hundred pixels of the original behavior image, fifty pixels correspond one-to-one with sensors 102-1 to 102-n. Pixels corresponding to sensors that were triggered at that time can be filled with a first color (e.g., black), and pixels corresponding to sensors that were not triggered at that time can be filled with a second color (e.g., white).

[0011] The behavior recognition system 10 can be used to recognize multiple types of behaviors, such as going home, going out, cooking, washing dishes, eating, resting, sleeping, taking a bath, watching TV, etc. Depending on the behavior recognition system 10, the number of behaviors that can be recognized may also vary. For example, in one embodiment, the behavior recognition system 10 has a high recognition capability and can recognize ten different behaviors; in another embodiment, the behavior recognition system 10 has a lower recognition capability and can only recognize six different behaviors.

[0012] When multiple dissimilar behaviors trigger similar sensors, the corresponding behavior images may have excessively high similarity, potentially causing confusion in identifying these dissimilar behaviors based on the behavior images. For example, "leaving home" and "returning home" are two dissimilar behaviors, but they may trigger similar sensors, leading to the misidentification of "leaving home" as "returning home" or vice versa. The behavior recognition method proposed in this embodiment can effectively reduce the above-mentioned situation.

[0013] Please refer to Figure 2 for a flowchart illustrating a behavior recognition method according to an embodiment of the present invention. In one embodiment, a computer-readable medium including a plurality of computer-readable instructions can be used to implement the behavior recognition method. The computer-readable medium may be included in a storage unit 1041. When the computer-readable medium is executed by a processing unit 1043, the processing unit 1043 may execute the behavior recognition method. The behavior recognition method can be applied to a behavior recognition system 10, and the computer-readable medium may be stored in the storage unit 1041.

[0014] Step S201: Obtain an original behavior image corresponding to a first time. Details of the original behavior image can be found above. In one embodiment, the processing unit 1043 may access the storage unit 1041 to obtain the original behavior image.

[0015] Step S203: Determine an image feature based on a second action corresponding to a second time. The second time precedes the first time, for example, the second time precedes the first time by a sampling interval. In one embodiment, the first time and the second time are sampling times. For example, assuming the sampling interval is one second, the second time is one second before the first time. That is, the second action is the action corresponding to the second time identified by the processing unit 1043 according to this method. The image feature may include one or more pixels and may be in any form, such as text, pattern, color, etc. The number of image features may be the same as the number of actions that the action recognition system 10 can recognize. For example, the action recognition system 10 can recognize ten different actions, and these ten different actions correspond one-to-one to ten different image features. That is, each action can correspond to a unique image feature. In one embodiment, two different actions will not correspond to the same image feature. In one embodiment, step S203 is executed first, and then step S201 is executed.

[0016] Step S205: Integrate the original behavior image and image features to generate a behavior image with features. In one embodiment, the processing unit 1043 replaces a portion of the original behavior image with image features to generate a behavior image with features, that is, the size of the behavior image with features is equal to that of the original behavior image. In another embodiment, the processing unit 1043 attaches (or connects) image features to the original behavior image to generate a behavior image with features, that is, the size of the behavior image with features is larger than that of the original behavior image. To better understand, several practical examples are given below to illustrate how to integrate the original behavior image and image features. In one embodiment, the image feature is a 2x2 square with four pixels having a grayscale value. During integration, the processing unit 1043 replaces the pixels in the original behavior image that are not used to represent the sensor, such as one of the four corners, with the image feature to generate a behavior image with features. In another embodiment, the image feature is a row of pixels with a grayscale value. During integration, the processing unit 1043 connects the image feature with the top or bottom row of pixels of the original behavior image to generate a behavior image with features. In another embodiment, the image features are multiple pixels with an RGB value. During integration, the processing unit 1043 attaches the image features as the outline of the original behavioral image to the outer edge of the original behavioral image to generate a behavioral image with features. The above examples are for illustrative purposes only and are not intended to limit the invention.

[0017] To better understand, Figure 5 illustrates an example of integrating an original behavior image and image features according to an embodiment of the present invention. In this example, image feature 51 is a row of pixels and has a corresponding grayscale value according to the corresponding behavior. During integration, image feature 51 is concatenated below the original behavior image 50 to produce a behavior image 52 with features.

[0018] Step S207: Determine a first behavior corresponding to a first time based on the characteristic behavior image. In one embodiment, the computer-readable medium includes programs for implementing one or more programs, such as neural networks, which can be used to determine the first behavior corresponding to a first time based on the characteristic behavior image.

[0019] By means of the above method, image features representing the second action determined at a second time prior to the first time can be "added" to the action image obtained at the first time to generate a characteristic action image. In this way, the characteristic action image contains image features representing the previous action, which allows for further judgment based on the previous action when judging the current action, thereby improving accuracy. In particular, for the current action which may be one of two different actions that trigger similarity sensors, such as "going home" and "going out", by adding the previous action as a basis for judgment, the probability of misjudgment can be effectively reduced.

[0020] Figure 3 is a flowchart of a method for mapping image features to behaviors according to an embodiment of the present invention. The following description, in conjunction with Figure 3, illustrates how the mapping between image features and behaviors is generated. In one embodiment, the image features corresponding to the behaviors are determined from a plurality of candidate image features.

[0021] Step S301: Pair each behavior that the behavior recognition system can identify with other behaviors to form multiple behavior pairs, and calculate the similarity between the two behaviors in each behavior pair. In one embodiment, the similarity between two behaviors refers to a parameter generated by quantifying the overlap between the sensor triggered by one behavior and the sensor triggered by another behavior. That is, the similarity represents the degree of similarity / overlap between the sensors triggered by two behaviors. The higher the degree of similarity / overlap between the triggered sensors, the higher the similarity. In one embodiment, the similarity between two behaviors is calculated using cosine similarity. In another embodiment, any mathematical tool that can be used to quantify the degree of similarity between the sensors triggered by two behaviors can be applied to calculate the similarity.

[0022] For example, suppose the behavior recognition system can identify six types of behavior, including going home, leaving home, cooking, washing dishes, resting, and sleeping. Behavior pairs include pairing "going home" with the other five behaviors, pairing "leaving home" with the other four behaviors (pairing "going home" and "leaving home" can be unique), pairing "cooking" with the other three behaviors, and so on. Therefore, behavior pairs include [going home - leaving home], [going home - cooking], [going home - washing dishes], [going home - resting], [going home - sleeping], [leaving home - cooking], [leaving home - washing dishes], etc., and so on. When executing step S301, the similarity between the two behaviors in each behavior pair is calculated. In the above example, the similarity between "going home" and "leaving home", "going home" and "cooking", "going home" and "washing dishes", "going home" and "resting", "going home" and "sleeping", "leaving home" and "cooking", "leaving home" and "washing dishes", "leaving home" and "resting", "leaving home" and "sleeping", etc., and so on. The similarity percentage can be represented in a table as shown in Table 1 below: Table 1 go home Going out Cooking rice do the washing up rest sleep go home X 95% 10% 13% 43% 16% Going out 95% X 11% 12% 20% 14% Cooking rice 10% 11% X 87% 18% 9% do the washing up 13% 12% 87% X 6% 7% rest 43% 20% 18% 6% X 37% sleep 16% 14% 9% 7% 37% X

[0023] Step S303: Determine the configuration order of the behavior pairs based on the similarity of the corresponding behavior pairs. In one embodiment, the behavior pairs are configured in descending order of similarity. In the example above, the similarity from high to low is 95%, 87%, 43%, 37%, ..., 7%, 6%, so the configuration order of the behavior pairs is [Go home - Go out], [Cook - Wash dishes], [Go home - Rest], [Rest - Sleep], ..., [Sleep - Wash dishes], [Rest - Wash dishes].

[0024] Step S305: Based on the configuration order and the probability distribution of the preceding behavior of each behavior, each behavior is mapped to a unique image feature. In one embodiment, the preceding behavior of each behavior refers to the behavior that precedes the current behavior by one sampling interval (i.e., the previous time). In one embodiment, the probability distribution of the preceding behavior of each behavior can be obtained by observation and statistics. For example, observing 1000 instances of "going home" and analyzing the distribution of the preceding behavior before "leaving home" in those 1000 instances, assuming that 700 of the 1000 instances were preceded by "leaving home" and 300 by "going home," the probability distribution of the preceding behavior before "going home" is 70% for leaving home and 30% for going home. Observing 1000 instances of "leaving home" and analyzing the distribution of the preceding behavior before "leaving home," assuming that 540 of the 1000 instances were preceded by "resting," 310 by "washing dishes," and 150 by "going home," the probability distribution of the preceding behavior before "leaving home" is 54% for resting, 31% for washing dishes, and 15% for going home, and so on. In one embodiment, when configuring image features, behavior pairs that appear earlier in the configuration order are given higher priority. Taking the aforementioned example, the first behavior pair in the configuration order is [Going Home - Leaving Home] with a similarity of 95%, the second behavior pair is [Cooking Rice - Washing Dishes] with a similarity of 87%, and the third behavior pair is [Going Home - Resting Home] with a similarity of 43%. Therefore, when configuring image features, [Going Home - Leaving Home] is given the highest priority, followed by [Cooking Rice - Washing Dishes], then [Going Home - Resting Home], and so on. For details of step S305, please refer to the flowchart shown in Figure 4. This process can begin with the first behavior pair in the configuration order after determining the first behavior pair in the configuration order.

[0025] Step S401: Determine whether the behavior with the highest probability in the probability distribution of the preceding behavior of one behavior in the behavior pair already has a corresponding image feature. If yes, proceed to S403; if no, proceed to S405.

[0026] Step S403: Determine whether the behavior with the highest probability in the probability distribution of the preceding behavior of one behavior in the behavior pair already has a corresponding image feature. If yes, proceed to S407; if no, proceed to S405.

[0027] Step S405: The behavior is mapped to a candidate image feature that is not mapped among multiple candidate image features, and then the candidate image feature corresponding to the behavior is used as the image feature.

[0028] Step S407: Determine whether all behaviors have corresponding image features. If yes, end this process; if no, proceed to step S409.

[0029] Step S409: Determine whether the configuration order has reached the last position. If yes, execute S413; if no, execute S411.

[0030] Step S411: Consider the behavior of sorting the next position in the configuration order, and return to S401.

[0031] Step S413: Assign the behavior that does not have a corresponding image feature to a candidate image feature that is not matched in the candidate image features.

[0032] The following example illustrates the process in Figure 4, using the candidate image features as a column (or row) of pixels with different grayscale values ​​and the aforementioned example. In the candidate image features, grayscale values, for example, from 0 to 255, are equally divided into the number of behavior types that the behavior recognition system can recognize. For example, if six behaviors can be recognized, then they can be divided into six values: 0, 51, 102, 153, 204, and 255, that is, there are a total of six candidate image features. Furthermore, the two preceding behaviors of a behavior pair will be assigned to two different grayscale values ​​with the greatest possible difference, for example, assigned to the two currently largest and smallest grayscale values. First, consider the behavior pair [Going Home - Going Out]. Among the behaviors preceding "Going Home" in the behavior pair [Going Home - Going Out], the one with the highest probability is "Going Out". Among the behaviors preceding "Going Out" in the behavior pair [Going Home - Going Out], the one with the highest probability is "Rest". At this time, neither "Going Out" nor "Rest" has a corresponding image feature. Therefore, "Going Out" is mapped to the candidate image feature grayscale value 255 as the image feature corresponding to "Going Out", and "Rest" is mapped to the candidate image feature grayscale value 0 as the image feature corresponding to "Rest". Next, consider the second-ranked behavior pair [cooking rice - washing dishes] in the configuration order. Assume that the highest probability preceding "cooking rice" in this pair is "resting," and the highest probability preceding "washing dishes" in the pair is "cooking rice." At this point, "cooking rice" still lacks a corresponding image feature, but "resting" already has a corresponding image feature with a grayscale value of 0. Therefore, "cooking rice" is assigned to a candidate image feature grayscale value of 204, which is farther from "resting," as the corresponding image feature for "cooking rice." "Resting" is no longer assigned to any other image feature. This process is repeated from S401 to S409 until all behaviors—namely, "going home," "going out," "cooking rice," "washing dishes," "resting," and "sleeping"—are assigned to different grayscale values. It is worth noting that in this embodiment, the candidate image features are one-to-one corresponding to behaviors that the behavior recognition system can identify. Furthermore, the difference in grayscale value between two image features determined by the first action in the configuration order will be greater than the difference in grayscale value between two image features determined by actions that are not first in the configuration order. In one embodiment, for two actions in a pair of actions that are earlier in the configuration order, the grayscale value difference corresponding to the action with the highest probability will be greater, so as to more clearly distinguish the two dissimilar actions.

[0033] In this way, after obtaining the original behavior image, a column (or a row) of pixels with grayscale values ​​that represent the second behavior (i.e. the behavior that is judged to correspond to the previous time) are "added" to the original behavior image to generate a behavior image with features. The neural network can then more accurately determine what the current behavior is based on the added image features (i.e. the previous behavior).

[0034] In one embodiment, step S304 may be included between step S303 and step S305. In step S304, the configuration order is adjusted according to the frequency of occurrence of each behavior. For a specific behavior, the frequency of occurrence represents the number of times the user actually performs the behavior in multiple time periods. For example, for "sleeping", the frequency of occurrence of each behavior can be obtained by observing the number of times the user actually "sleeps" in 1000 time periods, and so on. In this way, the number of occurrences of each behavior in these 1000 times can be understood, thereby obtaining the frequency of occurrence of each behavior. For behaviors with a frequency of occurrence below a certain threshold (a specific threshold can be set as needed), misjudgment may occur when determining whether it is the behavior because the frequency of occurrence of the behavior is low (i.e., the probability of it actually being the behavior is low). Therefore, step S304 can adjust the order of one or more behavior pairs including behaviors with a frequency of occurrence below the specific threshold, for example, by adjusting them before the behavior pair with the highest similarity. In step S405, image features are assigned according to the adjusted configuration order. Taking the previous example, assuming that the frequency of "sleeping" is below a certain threshold, when executing step S304, the order of [sleeping-resting] with the highest similarity among the behavior pairs containing "sleeping" can be adjusted to precede [going home-leaving].

[0035] In one embodiment, in step S403, behavior pairs can be divided into multiple problem groups based on multiple similarity thresholds, and then the configuration order can be determined based on the problem groups and similarity. For example, three similarity thresholds are set to 70%, 50%, and 30%, and four problem groups are set as serious problems, minor problems, normal problems, and no problems. In this example, behavior pairs with a similarity of 71% to 100% will be configured into the serious problem group, behavior pairs with a similarity of 51% to 70% will be configured into the minor problem group, and so on. When determining the configuration order, the behavior pairs in the serious problems can be sorted first, then the behavior pairs in the minor problems can be sorted, and so on.

[0036] In one embodiment, the order of adjustment in step S403 can be based on a combination of occurrence frequency and similarity threshold. For example, four problem groups can be set as critical problems, major problems, minor problems, and no problems, and two similarity thresholds can be set as 70% and 50%, respectively. Problem groups with an occurrence frequency below a specific threshold are configured as critical problems, behavior pairs with a similarity of 71% to 100% are configured as major problems, behavior pairs with a similarity of 51% to 70% are configured as minor problems, and so on. When determining the configuration order, behavior pairs in the critical problems can be sorted first, followed by behavior pairs in the major problems, minor problems, and no problems in that order.

[0037] It should be noted that the execution order of the steps shown in Figure 2 is only an example and is not limited. In actual applications, the execution order of these steps can be changed as needed. For example, the second time of step S203 is earlier than the first time of step S201. The execution order of the steps in Figure 4 is also not limited.

[0038] In the above embodiments, when configuring image features for each action pair, the action with the highest probability in the probability distribution of the preceding action is considered. However, in an alternative embodiment, when configuring image features for each action pair, the n actions with the top n highest probabilities in the probability distribution of the preceding action can be considered, where n is an integer greater than 1. For example, in one embodiment, the action pair ranked first in the configuration order is considered first, the top two highest probabilities in the probability distribution of the preceding action for each of the two actions in the action pair are found, and an image feature is configured for each of these four actions. Then, the action pair ranked second in the configuration order is considered, and so on.

[0039] Through an embodiment of the present invention, the misjudgment that is prone to occur when dissimilar behaviors trigger similarity sensors can be effectively reduced. This, in turn, improves the accuracy and security of the behavior recognition system.

[0040] In summary, although the present invention has been disclosed above by way of embodiments, it is not intended to limit the present invention. Those skilled in the art to which this invention pertains can make various modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of this invention shall be determined by the appended claims. [Simplified Explanation of the Diagram]

[0006] Figure 1 is a block diagram of a behavior recognition system according to an embodiment of the present invention. Figure 2 is a flowchart of a behavior recognition method according to an embodiment of the present invention. Figure 3 is a flowchart of a method for mapping image features to behaviors according to an embodiment of the present invention. Figure 4 is a flowchart of mapping each behavior to a unique image feature according to the configuration order and the probability distribution of the preceding behavior of each behavior. Figure 5 illustrates an example of integrating the original behavior image and image features according to an embodiment of the present invention.

Claims

1. A behavior recognition method, applicable to a behavior recognition system for recognizing a plurality of behaviors, comprising: Obtain an original behavior image corresponding to a first time point, the original behavior image including a plurality of pixels corresponding to whether a plurality of sensors are triggered; determine an image feature based on a second behavior corresponding to a second time point, the second time point preceding the first time point; integrate the original behavior image and the image feature to generate a behavior image with features; and determine a first behavior corresponding to the first time point based on the behavior image with features.

2. The behavior identification method as described in claim 1, wherein the image features are determined from a plurality of candidate image features, the candidate image features corresponding one-to-one with the behaviors, and a correspondence between the candidate image features and the behaviors is generated by: pairing each behavior with the other behaviors to form a plurality of behavior pairs, and calculating a similarity corresponding to each behavior pair; determining an arrangement order based on the similarity; and mapping the behaviors to the candidate image features based on the arrangement order and the probability distribution of the occurrence of a preceding behavior of each behavior.

3. The behavior identification method as described in claim 2 further includes determining the behavior pair that appears first in the configuration order, wherein for each behavior pair, mapping the behaviors to the candidate image features according to the configuration order and the probability distribution of the preceding behavior of each behavior includes: If the behavior with the highest probability in the probability distribution of the preceding behavior of one behavior in the behavior pair does not have a corresponding candidate image feature, then the behavior with the highest probability in the probability distribution of the preceding behavior of one behavior in the behavior pair is assigned to one of the candidate image features; and if the behavior with the highest probability in the probability distribution of the preceding behavior of another behavior in the behavior pair does not have a corresponding candidate image feature, then the behavior with the highest probability in the probability distribution of the preceding behavior of another behavior in the behavior pair is assigned to the other of the candidate image features.

4. The behavior recognition method as described in claim 2, wherein the candidate image features have different grayscale values.

5. The behavior identification method as described in claim 2, wherein a grayscale difference between the first candidate image feature and the second candidate image feature determined by the behavior pair that is ranked first in the configuration order is greater than a grayscale difference between the first candidate image feature and the second candidate image feature determined by the behavior pairs that are not ranked first in the configuration order.

6. The behavior recognition method as described in claim 2, further comprising, after determining an arrangement order based on the similarities: Adjust the configuration order according to the frequency of occurrence of each of these behaviors.

7. The behavior identification method as described in claim 6, wherein adjusting the configuration order based on the occurrence frequency of each behavior involves moving one or more behaviors, including behaviors whose occurrence frequency is below a specific threshold, forward in the order of the configuration.

8. The behavior identification method as described in claim 2, wherein when determining the configuration order based on the similarity, the behavior pairs are divided into multiple question groups based on the similarity and multiple similarity thresholds, and then the configuration order is determined based on the question groups and the similarity.

9. A computer-readable medium, when executed by a processing unit of a behavior recognition system for identifying a plurality of behaviors, causes the processing unit to perform: acquiring an original behavior image corresponding to a first time, the original behavior image including a plurality of pixels corresponding to whether a plurality of sensors are triggered; determining an image feature based on a second behavior corresponding to a second time, the second time preceding the first time; integrating the original behavior image and the image feature to generate a behavior image having features; and determining a first behavior corresponding to the first time based on the behavior image having features.

10. The computer-readable medium as described in claim 9, wherein the image features are determined from a plurality of candidate image features, which correspond one-to-one with the actions, and a correspondence between the candidate image features and the actions is generated by: pairing each action with the other actions to form a plurality of action pairs, and calculating a similarity for each action pair; determining an arrangement order based on the similarities; and mapping the actions to the candidate image features based on the arrangement order and a probability distribution of the occurrence of a preceding action of each action.

11. The computer-readable medium as described in claim 10 further includes determining the pair of actions that are first in the configuration order, wherein for each pair of actions, mapping the actions to the candidate image features according to the configuration order and the probability distribution of the preceding action of each action includes: If the behavior with the highest probability in the probability distribution of the preceding behavior of one behavior in the behavior pair does not have a corresponding candidate image feature, then the behavior with the highest probability in the probability distribution of the preceding behavior of one behavior in the behavior pair is assigned to one of the candidate image features; and if the behavior with the highest probability in the probability distribution of the preceding behavior of another behavior in the behavior pair does not have a corresponding candidate image feature, then the behavior with the highest probability in the probability distribution of the preceding behavior of another behavior in the behavior pair is assigned to the other of the candidate image features.

12. The computer-readable medium as described in claim 10, wherein the candidate image features have different grayscale values.

13. The computer-readable medium as described in claim 10, wherein a grayscale difference between the first candidate image feature and the second candidate image feature determined by the action that is first in the configuration order is greater than a grayscale difference between the first candidate image feature and the second candidate image feature determined by actions that are not first in the configuration order.

14. The computer-readable medium as described in claim 10, further comprising, after determining an arrangement order based on the similarities: The configuration order is adjusted based on the probability of each of these behaviors occurring.

15. The computer-readable medium as described in claim 14, wherein adjusting the configuration order according to a frequency of occurrence of each of the acts involves moving one or more acts that include acts with a frequency of occurrence below a certain threshold forward in the configuration order.

16. The behavior of a computer-readable medium as described in claim 10, wherein when determining the configuration order based on the similarities, the behavior pairs are divided into a plurality of problem groups based on the similarities and a plurality of similarity thresholds, and then the configuration order is determined based on the problem groups and the similarities.

Images