User action recognition system and method based on wearable terminal

Abstract

The embodiment of the invention provides a user action recognition system based on a wearable terminal. The system comprises a motion monitoring device, a data acquisition assembly, a positioning assembly, a communication assembly, an alarm assembly and a microprocessor. The motion monitoring equipment obtains motion data of the wearer; the data acquisition assembly acquires environment audio; the positioning assembly obtains position data of a wearer and determines a current area corresponding to the wearer; the communication assembly realizes data interaction among the motion monitoring equipment, the data acquisition assembly, the positioning assembly, the alarm assembly and the microprocessor; the alarm assembly sends a prompt to the wearer; the microprocessor determines a motion mapping sequence of the wearer based on the motion data, the environment audio and the current area; based on the action mapping sequence, the microprocessor determines an action evaluation result of the wearer; when the action evaluation result does not meet a preset condition, the microprocessor sends a prompt to the wearer through the alarm assembly, and the preset condition is related to whether the action evaluation result is qualified or not.

Description

Technical Field

[0001] The present invention relates to the field of motion recognition technology, and in particular to a user motion recognition system and method based on a wearable terminal. Background Art

[0002] Offline merchants often face various challenges when managing their stores, and the standardized management of store employees is extremely important. During the operation process, store employees may ignore the standardization of service processes for various reasons, resulting in a variety of irregular operations. If these situations are not taken seriously, they will seriously affect the customer experience and have an adverse impact on the brand image. How to effectively manage the standardization of store employees has become a difficult point in store management.

[0003] Therefore, it is hoped to provide a user motion recognition system and method based on a wearable terminal, so as to better manage the store. Summary of the invention

[0004] One of the embodiments of this specification provides a user motion recognition system based on a wearable terminal. The system includes: a motion monitoring device, a data acquisition component, a positioning component, a communication component, an alarm component and a microprocessor. The motion monitoring device is configured to obtain the motion data of the wearer; the data acquisition component is configured to collect ambient audio; the positioning component is configured to obtain the position data of the wearer and determine the current area corresponding to the wearer; the communication component is configured to realize data interaction between the motion monitoring device, the data acquisition component, the positioning component, the alarm component and the microprocessor; the alarm component is configured to send a reminder to the wearer; the microprocessor is configured to determine the motion mapping sequence of the wearer based on the motion data, the ambient audio and the current area; based on the motion mapping sequence, determine the motion evaluation result of the wearer; in response to the motion evaluation result not meeting the preset condition, send the reminder to the wearer through the alarm component.

[0005] One of the embodiments of the present specification also provides a user motion recognition method based on a wearable terminal, the method comprising obtaining motion data of the wearer; collecting ambient audio; obtaining location data of the wearer, and determining a current area corresponding to the wearer; realizing data interaction; sending a reminder to the wearer; determining a motion mapping sequence of the wearer based on the motion data, the ambient audio and the current area; determining a motion evaluation result of the wearer based on the motion mapping sequence; and in response to the motion evaluation result not meeting a preset condition, sending the reminder to the wearer through the alarm component. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0007] Figure 1 is a schematic diagram of a scenario of a user motion recognition system based on a wearable terminal according to some embodiments of this specification;

[0008] Figure 2 is an exemplary structural diagram of a user motion recognition system based on a wearable terminal according to some embodiments of this specification;

[0009] Figure 3 is an exemplary flow chart of a method for identifying user actions based on a wearable terminal according to some embodiments of this specification;

[0010] Figure 4 is a schematic diagram of determining an action mapping sequence according to some embodiments of this specification;

[0011] Figure 5 It is a schematic diagram of determining action evaluation results according to some embodiments of this specification. DETAILED DESCRIPTION

[0012] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0013] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0014] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0015] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0016] Offline merchants often face various challenges when managing their stores, and the standardized management of store employees is extremely important. During the operation process, store employees may ignore the standardization of service processes for various reasons, resulting in a variety of irregular operations. If these situations are not taken seriously, they will seriously affect the customer experience and have an adverse impact on the brand image. How to effectively manage the standardization of store employees has become a difficult point in store management.

[0017] Some embodiments of this specification provide a user motion recognition system and method based on a wearable terminal, so as to better manage stores.

[0018] Figure 1 It is a scenario diagram of a user motion recognition system based on a wearable terminal according to some embodiments of this specification.

[0019] like Figure 1 As shown, the components in the application scenario 100 of the wearable terminal-based user motion recognition system may include an administrator terminal 110, a server 120, a storage device 130, a network 140, and a wearable terminal 150. In some embodiments, the components in the application scenario 100 may be connected and / or communicated with each other via the network 140 (e.g., wireless connection, wired connection, or a combination thereof).

[0020] The user action recognition method based on wearable terminals provided in some embodiments of this specification can be applied to a variety of scenarios where user actions need to be recognized. Exemplarily, the method can be applied to store management scenarios, where store employees wear wearable terminals, collect employee motion data, determine the actions performed by employees, and evaluate the actions performed by employees, thereby determining whether the actions performed by employees meet the requirements, so as to achieve the purpose of daily management of the store's offline business. In some embodiments, the method can also be applied to other scenarios where user actions need to be recognized, such as judging the standardization of production operations of workers in factories, judging the standardization of operations of medical staff in medical scenarios, etc.

[0021] In some embodiments, the user action recognition system based on the wearable terminal can be used to identify the user action of the wearer corresponding to the wearable terminal in the store, and determine whether the user action complies with the specification. When the terminal user participates in the network, the user action recognition system based on the wearable terminal can obtain the judgment result of whether the aforementioned user action complies with the specification.

[0022] The administrator terminal 110 refers to one or more terminals or software with network connection function used by the administrator user. In some embodiments, the administrator user can access the network through the administrator terminal 110, conduct network activities, and complete data transmission. In some embodiments, the administrator terminal 110 can be a mobile device, a tablet computer, a laptop computer, or any combination of other devices with input and / or output functions.

[0023] In some embodiments, the terminal user may be a store manager, and the wearer of the wearable terminal may be a store clerk. For example, when the terminal user participates in the network, the user action recognition system based on the wearable terminal can be used to determine whether the actions of the clerk in the store corresponding to the wearable terminal are standardized, so as to better manage the store. For another example, when the wearer of the wearable terminal participates in the network, the user action recognition system based on the wearable terminal can be used to determine whether his own actions are standardized, so as to adjust irregular actions in time and ensure work quality.

[0024] The server 120 may process data and / or information related to the wearable terminal-based user motion recognition system. In some embodiments, the server 120 accesses information and / or data from the storage device 130, the administrator terminal 110, and / or the wearable terminal 150. In some embodiments, the server may obtain the wearer's motion data from the storage device 130, the administrator terminal 110, and / or the wearable terminal 150.

[0025] The server 120 may be used to manage resources and process data and / or information from at least one component of the system or an external data source (eg, a cloud data center). In some embodiments, the server may determine a motion evaluation result of the wearer.

[0026] In some embodiments, the server 120 may include a processing device 121. The processing device 121 may process data and / or information obtained from other devices or system components. The processing device 121 may execute program instructions based on these data, information and / or processing results to perform the functions described in one or more embodiments of this specification.

[0027] In some embodiments, the processing device 121 may be configured as a microprocessor. For more information about the microprocessor, see the specification. Figure 2 Related description in .

[0028] The storage device 130 can be used to store data and / or instructions. The storage device 130 is implemented in a single central server, multiple servers connected by a communication link, or multiple personal devices. In some embodiments, the storage device 130 can also be used to provide one or more sources of other information for the user motion recognition system based on the wearable terminal. The storage device 130 can be one or more devices, one or more application programming interfaces, one or more protocol-based information acquisition interfaces, or other ways to obtain information. The information provided by the storage device 130 may already exist when the information is extracted, or it may be temporarily generated when the information is extracted, or it may be a combination of the above methods. In some embodiments, the storage device 130 can store the wearer's motion data, motion mapping sequences, ambient audio of the store, etc.

[0029] The network 140 can connect the various components of the system and / or connect the system with external resources. The network 140 allows the various components to communicate with each other and with other parts outside the system, and promotes the exchange of data and / or information. In some embodiments, the network 140 may include a local area network and other networks within the venue. For example, the network 140 may include a local area network within the venue to which the administrator terminal 110 and / or the wearable terminal 150 requests access. For example, when the administrator terminal 110 and / or the wearable terminal 150 access the network 140, the user can perform network activities through the administrator terminal 110 and / or the wearable terminal 150, thereby transmitting data.

[0030] In some embodiments, the network 140 may also include a cable network, a fiber optic network, a telecommunications network, the Internet, or any combination thereof. The network connection between the various parts may be in one of the above-mentioned ways or in multiple ways. In some embodiments, the network may be a point-to-point, shared, centralized, or other topological structure or a combination of multiple topological structures. In some embodiments, the network 140 may include one or more network access points. For example, the network 140 may include a wired or wireless network access point, such as a base station and / or a network exchange point, through which one or more components of the user motion recognition system based on the wearable terminal may be connected to the network 140 to exchange data and / or information.

[0031] The wearable terminal 150 refers to one or more portable terminals with network connection function used by the wearer user. In some embodiments, the wearer user can access the network through the wearable terminal 150, conduct network activities, and complete data transmission. In some embodiments, the wearable terminal 150 can be one or any combination of a wristband, a watch, smart glasses, a headset, or other devices with input and / or output functions.

[0032] In some embodiments, the wearable terminal 150 may include one or more of a motion monitoring device, a data acquisition component, a positioning component, a communication component, and an alarm component. For detailed descriptions of the motion monitoring device, the data acquisition component, the positioning component, the communication component, and the alarm component, please refer to this specification. Figure 2 Related description in .

[0033] This specification uses a restaurant store as an example to illustrate a user action recognition system and method based on a wearable terminal. It is understandable that the user action recognition system and method based on a wearable terminal involved in one or more embodiments of this specification can be used not only in restaurant stores, but also in other scenarios where user actions need to be recognized.

[0034] It should be noted that the application scenarios of the user motion recognition system based on the wearable terminal in some embodiments of this specification are provided for illustrative purposes only and are not intended to limit the scope of this specification. For those of ordinary skill in the art, various modifications or variations can be made according to the description of this specification. However, the changes and modifications will not deviate from the scope of this specification.

[0035] Figure 2 This is an exemplary structural diagram of a user motion recognition system based on a wearable terminal according to some embodiments of this specification.

[0036] like Figure 2 As shown, in some embodiments, the user motion recognition system 200 based on the wearable terminal may include a motion monitoring device 210, a data acquisition component 220, a positioning component 230, a communication component 240, an alarm component 250, and a microprocessor 260.

[0037] The motion monitoring device 210 is a device used to detect the motion state of the wearer and obtain the wearer's motion data.

[0038] In some embodiments, the motion monitoring device 210 can obtain the wearer's motion data in real time. The wearer can be a restaurant employee wearing a wearable terminal, and the wearer's physical signs can include gender, age, height, weight, etc., which can be manually input. The wearable terminal can include a terminal device such as a smart watch. For more information about motion data, see Figure 3 and related descriptions.

[0039] In some embodiments, the motion monitoring device 210 can be configured as different types of sensors, such as an accelerometer, a gyroscope, a magnetometer, and a heart rate sensor, etc., wherein the accelerometer can measure the acceleration and motion of an object, and the gyroscope can measure the rotation rate and angular direction of an object.

[0040] The data acquisition component 220 refers to a device used to collect, measure and record various environmental parameters.

[0041] In some embodiments, the data collection component 220 can collect ambient audio, which can include at least one of human voice and operation sound. Taking a restaurant as an example, there are sounds of eating and talking in the dining area, and sounds of cooking in the kitchen. In some embodiments, the data collection component 220 can include an audio collector, such as a microphone.

[0042] The positioning component 230 refers to a module for determining the position of a person in space.

[0043] In some embodiments, the positioning component 230 can obtain the wearer's location data and determine the current area corresponding to the wearer, such as the front hall, back kitchen, etc. In some embodiments, the positioning component 230 can include a variety of positioning tools, such as a Wi-Fi receiver, a light sensor, etc.

[0044] The communication component 240 refers to a module used to implement data transmission between devices.

[0045] In some embodiments, the communication component 240 can realize data interaction between the motion monitoring device 210, the data acquisition component 220, the positioning component 230, the alarm component 250 and the microprocessor 260. In some embodiments, the communication component 240 can be implemented through different signal methods, such as Wi-Fi, Bluetooth, cellular network, etc.

[0046] The alarm component 250 refers to a device for issuing an alarm or notification when a preset alarm condition is met.

[0047] In some embodiments, in response to the alarm instruction sent by the microprocessor 260, the alarm component 250 can send a reminder to the wearer. In some embodiments, the alarm component 250 can include different alarm reminder modes, such as sound, vibration, light, etc.

[0048] In some embodiments, the alarm component 250 can be provided on a wearable terminal, an administrator terminal, and / or configured separately.

[0049] The microprocessor 260 refers to a module for processing data, executing instructions, and controlling other components.

[0050] In some embodiments, the microprocessor 260 can determine the wearer's motion mapping sequence based on motion data, ambient audio and current area; determine the wearer's motion evaluation result based on the motion mapping sequence; and send a reminder to the wearer through an alarm component in response to the motion evaluation result not meeting a preset condition.

[0051] In some embodiments, the microprocessor is further configured to determine a load index based on the audio environment and the current area; and determine an action mapping sequence based on the load index and the motion data.

[0052] In some embodiments, the microprocessor is further configured to determine an action probability distribution based on wearer information of the wearer; and determine an action mapping sequence based on the motion data, the action probability distribution and the load index.

[0053] In some embodiments, the microprocessor is further configured to determine an action mapping subsequence from the action mapping sequence, and determine an action evaluation result based on the action mapping subsequence, customer flow data in a preset time period, and a stage order quantity.

[0054] In some embodiments, the microprocessor is further configured to determine the action standard based on the action mapping sequence; determine the action execution degree based on the customer flow data, the stage order quantity and the action mapping subsequence; and determine the action evaluation result according to preset rules based on the action standard and the action execution degree.

[0055] In some embodiments, the microprocessor 260 may include, but is not limited to, a central processing unit (CPU), a field programmable gate array (FPGA), and the like.

[0056] In some embodiments, the microprocessor 260 may be configured in the wearable terminal 150 or may be independently configured.

[0057] For more information about the functions performed by each module of the wearable terminal-based user motion recognition system 200, please refer to this specification. Figure 3-Figure 5 and related instructions.

[0058] It should be noted that the above description of the user motion recognition system based on the wearable terminal and its modules is only for the convenience of description and cannot limit this specification to the scope of the embodiments. It is understandable that for those skilled in the art, after understanding the principle of the system, it is possible to arbitrarily combine the modules or form a subsystem to connect with other modules without deviating from this principle. In some embodiments, Figure 2 The components disclosed in the specification may be different components in a system, or one component may realize the functions of two or more components. Such variations are within the protection scope of this specification.

[0059] In some embodiments, when implementing a user motion recognition method based on a wearable terminal, the microprocessor can determine the wearer's motion mapping sequence based on motion data, ambient audio and current area; determine the wearer's motion evaluation result based on the motion mapping sequence; in response to the motion evaluation result not meeting a preset condition, send a reminder to the wearer through an alarm component.

[0060] Figure 3 is an exemplary flow chart of a method for identifying user actions based on a wearable terminal according to some embodiments of this specification. Figure 3 As shown, the process 300 includes the following steps. In some embodiments, the process 300 can be executed by a microprocessor.

[0061] Step 310, determining the motion mapping sequence of the wearer based on the motion data, the ambient audio and the current area.

[0062] The wearer is the user of the wearable terminal. For more information about wearable terminals, see Figure 1 Related instructions.

[0063] In some embodiments, the wearer may be a staff member of a store. Taking a restaurant store as an example, the wearer may be one or more of a waiter, a cook, a cashier, or other store staff. For different types of stores, the wearer may correspond to different personnel.

[0064] Motion data refers to data related to the movement of the wearer. Motion data may include one or more types of data that can characterize the wearer's movement. For example, the motion data may include but is not limited to one or more of the wearer's movement speed, heart rate, and steering data. Among them, the steering data may include the wearer's steering direction, steering speed, steering angle, etc. In some embodiments, the motion data may be sequence data, including at least one type of data within a period of time.

[0065] In some embodiments, the microprocessor can monitor the wearer's motion data in real time based on the motion monitoring device. The motion monitoring device can be configured on the wearable terminal corresponding to the wearer. For more information about the motion monitoring device, see Figure 1 Related instructions.

[0066] In some embodiments, the frequency with which the motion monitoring device acquires motion data is related to at least one of the store's traffic information and order quantity.

[0067] In some embodiments, the microprocessor can obtain the store's traffic data and order quantity based on a variety of methods. For example, the microprocessor can obtain the store's traffic data by obtaining user input, infrared sensing device technology, video analysis technology, AI scene recognition, etc., and obtain the store's order quantity by reading data from store systems such as backend systems and cash register systems and performing statistics on them.

[0068] In some embodiments, the microprocessor may also obtain the store's customer flow information and order quantity through other feasible methods.

[0069] In some embodiments, the acquisition frequency is positively correlated with both the flow of people information and the number of orders. The more people flow in the store and the more orders there are, the higher the frequency with which the motion monitoring device acquires motion data.

[0070] The more people flow and the more orders there are, the greater the workload of store employees, and the more frequent their movements will be. By increasing the frequency of acquiring motion data, it is possible to more effectively monitor employees' motion data and obtain more comprehensive motion data, which is conducive to providing better data support for subsequent user motion recognition.

[0071] Ambient audio refers to various sounds in the environment the wearer is in. Taking a restaurant as an example, the ambient audio may include but is not limited to the voices of employees and / or customers, and one or more of the operating sounds in the environment (e.g., the sounds of lighting a fire, cooking, and packing food).

[0072] The ambient audio will be different depending on the environment the wearer is in. For example, when the wearer is at the door, the ambient audio will include more outdoor noise, when the wearer is in the kitchen, the ambient audio will include the sound of cooking, and when the wearer is in the dining area, the ambient audio will include the sound of chewing and the collision of tableware, etc. In different environments, the wearer will perform different actions. By collecting the ambient audio and determining the action mapping sequence based on the ambient audio, it is helpful to obtain a more accurate action mapping sequence.

[0073] The current area refers to the area where the wearer is currently located. In some embodiments, the microprocessor can obtain the wearer's location data through the positioning component and determine the current area through the location data. For more information, please refer to this manual Figure 2 Related description in .

[0074] Area refers to at least one partition obtained by dividing the store space. In some embodiments, the store can be divided into multiple areas for performing different functions based on the functional structure. For example, a restaurant store can be divided into different areas such as the kitchen, cashier counter, and dining-in area based on the functional structure. In some embodiments, the area can also be divided in other ways, and the area division method of different stores may be different, which can be determined based on actual needs.

[0075] The action mapping sequence refers to a sequence of at least one action performed by the wearer arranged in chronological order. In some embodiments, the action mapping sequence also includes the duration and area of ​​occurrence of the action performed by the wearer.

[0076] In some embodiments, the aforementioned at least one action may include at least one common action of the wearer. Taking a restaurant as an example, the actions performed by the wearer include at least one of washing hands, cleaning tables, welcoming guests, taking orders, and clearing tables. For different wearers, the common actions corresponding to the wearers are different. For different stores, the common actions corresponding to the wearers are also different.

[0077] In some embodiments, the action mapping sequence includes at least one target action arranged in chronological order, and its corresponding duration and occurrence area. For example, the action mapping sequence may include at least one element for representing the target action arranged in chronological order according to the occurrence time, each element is composed of a number of sub-elements, and the sub-elements may include but are not limited to the action name, duration and occurrence area.

[0078] In some embodiments, the microprocessor can determine the motion mapping sequence of the wearer based on the motion data of the wearer, the current area where the wearer is located, and the ambient audio. Exemplarily, the microprocessor can construct a vector to be matched based on the motion data of the wearer and the ambient audio, search the reference vector library corresponding to the current area based on the vector to be matched, and determine the reference motion mapping sequence corresponding to the reference vector with the highest similarity to the vector to be matched as the motion mapping sequence of the wearer. The similarity can be determined based on the vector distance, and the vector distance includes but is not limited to the Euclidean distance, the cosine distance, etc.

[0079] The microprocessor can construct a reference vector library based on historical data, and different areas correspond to different reference vector libraries, and each reference vector library includes at least one reference action mapping sequence. For example, the microprocessor can construct a reference vector based on the historical motion data and historical environmental audio of the historical wearer in the current area, and determine the historical action mapping sequence corresponding to the historical motion data and historical environmental audio as the reference action mapping sequence corresponding to the reference vector. Among them, the historical action mapping sequence can be determined based on the user's input. For example, the administrator terminal can send historical motion data and its corresponding image data to the administrator user, and obtain user input, and the user input includes the historical action mapping sequence corresponding to the historical motion data determined by the user based on the historical motion data and its corresponding image data. The aforementioned image data can be video data corresponding to the historical motion data.

[0080] In some embodiments, the microprocessor can also determine the load index based on the ambient audio and the current area; determine the action mapping sequence based on the load index and the motion data. For more information, please refer to this manual. Figure 4 and its description.

[0081] Step 320, determining the motion evaluation result of the wearer based on the motion mapping sequence.

[0082] The action evaluation result refers to the evaluation result of the standardization degree of the action performed by the wearer. In some embodiments, the action evaluation result may include qualified or unqualified. Among them, the action evaluation result is qualified, which indicates that the standardization degree of the action in the action mapping sequence meets the requirements.

[0083] In some embodiments, the microprocessor can calculate multiple similarities between the action mapping sequence and multiple standard action mapping sequences in the standard sequence library to determine the aforementioned multiple action evaluation results. For example, if the maximum value of the multiple similarities is greater than the similarity threshold, the evaluation result is qualified, otherwise, the evaluation result is unqualified. The similarity threshold can be a system default value or can be manually set.

[0084] Different regions correspond to different standard sequence libraries, and each standard sequence library includes a standard action mapping sequence corresponding to the operations that may occur in the region. The standard action mapping sequence is a data sequence used to characterize standard actions, which may include at least one standard action arranged in chronological order. In some embodiments, the standard action mapping sequence can be determined based on user input.

[0085] In some embodiments, the microprocessor can also determine an action mapping subsequence from the action mapping sequence, and determine the action evaluation result based on the action mapping subsequence, the passenger flow data of the preset time period, and the stage order volume of the preset time period. For more relevant instructions, please refer to this manual. Figure 5 and its related description.

[0086] Step 330, in response to the action evaluation result not meeting the preset condition, sending a reminder to the wearer through the alarm component.

[0087] The preset condition is a pre-set condition for determining whether to send a reminder to the wearer. In some embodiments, the preset condition can be that the action evaluation result is qualified.

[0088] When the wearer's motion evaluation result is unqualified, the motion evaluation result does not meet the preset conditions. At this time, the microprocessor can send a reminder to the wearer through the alarm component. The reminder can be in a variety of ways, for example, the reminder can be in the form of text, voice, vibration, etc. For more information about the alarm component, please refer to this manual Figure 2 Related description in .

[0089] In some embodiments of this specification, the actions performed by employees are monitored through wearable terminals worn by employees, so that abnormal situations such as unqualified actions of employees at work can be discovered in time, which is helpful for store management and improving service quality.

[0090] Figure 4 FIG. 1 is a schematic diagram of determining an action mapping sequence according to some embodiments of this specification. Figure 4 As shown, the microprocessor can determine a load index 430 based on the ambient audio 410 and the current area 420; and determine an action mapping sequence 490 based on the load index 430 and the motion data 440. For more information about ambient audio, current area, load index, motion data, and action mapping sequence, see Figure 3 Related instructions.

[0091] The load index is data used to reflect the busyness of the wearer in the current area. In some embodiments, the load index can be expressed as a numerical value. The higher the load index, the higher the busyness of the wearer in the current area.

[0092] In some embodiments, the microprocessor can determine the load index based on the flow of people information and the number of orders. The load index is positively correlated with both the flow of people information and the number of orders. For example, the microprocessor can use the number of orders per person per unit time and / or the number of customers served per person per unit time as the load index. For more information about flow of people information and order quantity, see Figure 3 Related instructions.

[0093] In some embodiments, the microprocessor can determine the load index based on the ambient audio, motion data, and current area. For example, the microprocessor can construct a vector to be matched based on the ambient audio and motion data, search the historical database based on the vector to be matched, obtain the historical data vector corresponding to the same area as the vector to be matched and with the greatest similarity, and determine the label corresponding to the historical data vector as the load index. The similarity can be determined based on the vector distance, and the vector distance includes but is not limited to the Euclidean distance, the cosine distance, etc.

[0094] The historical database can be constructed based on the historical data. The historical database includes multiple historical data vectors and corresponding regions and labels. The elements of the historical data vector include historical environmental audio and historical motion data. The label is the historical load index of the wearer corresponding to the historical data vector, and the label can be obtained by manual annotation.

[0095] In some embodiments, the microprocessor may determine the action mapping sequence 490 based on the load index 430 and the motion data 440. For example, the microprocessor may determine at least one preset action corresponding to the load index and at least one preset action feature corresponding to the load index by querying the reference motion data table corresponding to the current area based on the load index; divide the motion data into at least one motion sub-data based on a preset division rule; compare each motion sub-data with the aforementioned at least one preset action feature, determine the preset action with the highest similarity as the target action corresponding to the motion sub-data, and use the duration corresponding to the motion sub-data as the duration of the target action, and use the area where the motion sub-data is collected as the occurrence area of ​​the target action; based on the comparison result of at least one motion sub-data with at least one preset action feature, determine at least one group of target action data, each group of target action data includes the action name, duration and occurrence area of ​​the target action, determine at least one group of target action data as at least one element in the action mapping sequence, and sort them by occurrence time, thereby obtaining the action mapping sequence.

[0096] In some embodiments, the reference motion data table may include a reference load index, at least one reference motion corresponding to the reference load index, and at least one reference motion feature corresponding thereto.

[0097] In some embodiments, the reference motion data table is constructed based on historical data, and each area in the store has a corresponding reference motion data table. For example, the microprocessor can count the actions that occur in each area under different load indexes in the historical data, and for each area, determine the action with a frequency greater than a preset frequency threshold as a reference action, and analyze the reference action to determine the reference action feature corresponding to the reference action; and construct the reference motion data table corresponding to each area based on the reference action corresponding to the area, the reference action feature, and the load index corresponding to the reference action.

[0098] The reference action feature refers to an action included in at least one execution step corresponding to the reference action. In some embodiments, the reference action feature can be obtained by analyzing the reference action based on an image analysis algorithm, or determined based on user input.

[0099] In some embodiments, the preset division rule may include dividing the motion data based on similar pause intervals in the motion data.

[0100] Similar pause intervals refer to intervals in which the value of the motion data is continuously 0 or the value is continuously less than the sensing threshold. The sensing threshold refers to the minimum sensing value of the motion amplitude. When the motion data is less than the sensing threshold, it can be judged that the wearer has not made any effective movement. No effective movement means that the wearer's movement amplitude is very small and has no other impact.

[0101] In some embodiments, the sensing threshold is related to the type of motion data and can be determined based on prior experience. Figure 3 Related description in .

[0102] For example, for a segment of motion data of speed type {10, 13, 23, 10, 11, 0, 0, 0, 10, 12, 9, 2, 3, 2, 3}, if the sensing threshold is 5, the two segments of data (0, 0, 0) and (2, 3, 2, 3) can be considered as similar pause intervals, and the microprocessor can divide the aforementioned motion data into two motion sub-data (10, 13, 23, 10, 11) and (10, 12, 9) based on these two similar pause intervals.

[0103] In some embodiments, the microprocessor may determine an action probability distribution 460 based on the wearer information 450 of the wearer; and determine an action mapping sequence 490 based on the motion data 440 , the action probability distribution 460 , and the load index 430 .

[0104] The wearer information refers to information related to the wearer's identity. In some embodiments, the wearer information includes at least the wearer's name and occupation (e.g., chef, waiter, cashier, etc.). The wearer information may also include other data, which may be determined according to actual needs.

[0105] The action probability distribution is data that characterizes the wearer's action characteristics, and may include at least one action that the wearer will perform and its corresponding probability.

[0106] In some embodiments, the microprocessor can determine the general action data based on the wearer information, and the general action data can include at least one general action commonly used in the type of work corresponding to the wearer information, and the general action features corresponding to the general action. For example, the microprocessor can obtain the historical data recorded by all wearable terminals, determine at least one high-frequency action corresponding to different types of work based on multiple historical wearer information and their corresponding historical action data in the historical data, and use the at least one high-frequency action as the general action of the type of work, and determine the general action features based on the high-frequency action features corresponding to the high-frequency action.

[0107] High-frequency actions refer to actions that appear in historical action data at a frequency higher than a preset frequency threshold. The preset frequency threshold can be set based on prior experience and / or actual needs.

[0108] Common actions refer to commonly applicable actions that wearers of the same job type will perform. Common actions are related to the job type. For example, common actions corresponding to the job type of chef may include but are not limited to at least one of preparing food, cooking, and plating.

[0109] In some embodiments, the microprocessor can obtain the wearer's personal historical data based on the wearer's name in the wearer information, determine the frequency of different actions in the personal historical data, filter out common actions belonging to the wearer's corresponding job type, and determine the action probability distribution based on the filtered actions and their corresponding frequencies.

[0110] In some embodiments, the microprocessor may determine an action mapping sequence based on the motion data 440 , the action probability distribution 460 , and the load index 430 .

[0111] Exemplarily, the microprocessor can divide the motion data into at least one motion sub-data based on a preset rule, compare each motion sub-data with the general action feature corresponding to the general action in the aforementioned action probability distribution, and determine the similarity between the motion sub-data and the general action; use the frequency corresponding to the general action in the action probability distribution as the occurrence probability corresponding to the general action; and based on the aforementioned similarity and occurrence probability, determine the evaluation value of the general action, determine the general action with the highest evaluation value as the target action corresponding to the motion sub-data, and use the duration corresponding to the motion sub-data as the duration of the target action, and use the area where the motion sub-data is collected as the occurrence area of ​​the target action; based on the comparison result of at least one motion sub-data with at least one general action feature in the action probability distribution, determine at least one group of target action data, each group of target action data includes the action name, duration and occurrence area of ​​the target action, determine at least one group of target action data as at least one element in the action mapping sequence, and sort them according to the occurrence time, so as to obtain the action mapping sequence.

[0112] In some embodiments, the evaluation value of the general action can be determined by the aforementioned similarity and occurrence probability based on a preset weighting coefficient.

[0113] For example, the microprocessor can calculate the evaluation value of the general action by the following formula (1): D=k 1 ×S+k 2 ×P (1)

[0114] Among them, D is the evaluation value of the general action, S is the similarity between the motion sub-data and the general action, P is the occurrence probability corresponding to the general action, k 1 , k 2 is the weight, which can be set based on prior experience, and k 1 +k 2 =1.

[0115] In some embodiments, k 1 Related to the load index, the larger the load index, the higher the k 1 The bigger.

[0116] In some embodiments, the microprocessor may also determine the motion mapping sequence 490 through the motion estimation model 480 based on the motion data 430 , the motion probability distribution 460 , and the load index 430 .

[0117] The action prediction model is a model for determining the action mapping sequence. In some embodiments, the action prediction model is a machine learning model, such as a deep neural network (DNN) model.

[0118] The input of the motion estimation model includes motion data, motion probability distribution and load index, and the output is a sequence of motion mappings.

[0119] For more information about motion data, motion probability distribution, load index, and motion mapping sequence, please refer to the previous description.

[0120] In some embodiments, the microprocessor can train the initial prediction model based on multiple labeled training samples to obtain a trained action prediction model. For example, the microprocessor can input multiple training samples into the initial prediction model, construct a loss function based on the output and label of the initial prediction model, iteratively update the parameters of the initial action prediction model based on the loss function, and the iterative update method includes but is not limited to the gradient descent method. When the iteration completion condition is met, the iteration is stopped to obtain the trained action prediction model. The iteration completion condition includes the convergence of the loss function, the number of iterations reaching a preset threshold, etc.

[0121] In some embodiments, the training sample may include multiple groups of sample data, each group of sample data may include sample motion data, sample action probability distribution and sample load index at a first historical time, and the training sample may be obtained based on historical data. The label corresponding to each group of sample data may be a historical actual action mapping sequence of the wearer corresponding to the group of sample data at a second historical time, and the label may be obtained by monitoring the wearer or by manual labeling. The first historical time and the second historical time may be determined based on actual conditions, and the second historical time may be later than the first historical time.

[0122] In some embodiments, the input to the action prediction model also includes order distribution data 470 .

[0123] The order distribution data characterizes the quantity distribution of orders with different progress. For example, the order stage includes but is not limited to the order receiving stage, order preparation stage, order service stage, order confirmation stage, etc., then the order distribution data can be [2, 1, 3, 4], indicating that 2 orders are in the order receiving stage, 1 order is in the order preparation stage, 3 orders are in the order service stage, and 4 orders are in the order confirmation stage. The above division of order stages is only an example for ease of understanding. In fact, the order stages can also be divided based on other methods, which can be set according to needs.

[0124] In some embodiments, the microprocessor can determine the stage of the order by reading the data of the wearable terminal, and count the number of orders at different stages, and then the order distribution data. Exemplarily, a standard processing flow can be pre-set in the wearable terminal, and the standard processing flow includes at least one order stage and a confirmation operation that needs to be performed by the wearer. For example, in response to receiving an order and starting processing, the wearer enters confirmation information on the page related to the order through the wearable terminal to determine that the order enters the processing stage. After each stage is completed, the wearer can submit confirmation information through the terminal. The stage corresponding to the order is determined by the confirmation information entered by the wearer and at least one order stage of the standard processing flow; the microprocessor can read the relevant data in the wearable terminal to determine the stage of the order.

[0125] In some embodiments, when the input of the action prediction model includes order distribution data, each set of sample data in the training sample may also include sample order distribution data of a first historical time, and the sample order distribution data may be acquired based on historical data.

[0126] Considering that some orders can be processed in parallel, employees may also process orders in parallel rather than in a single line. By considering the order distribution data, it is helpful to be more in line with the actual situation when predicting the action mapping sequence.

[0127] In some embodiments of the present specification, the action mapping sequence is determined by an action estimation model, so that the action mapping sequence can be determined quickly and accurately.

[0128] In some embodiments of the present specification, when determining the action probability distribution, both the employee's historical operating habits and the characteristics of different types of work are taken into consideration, so that the obtained action probability distribution is more in line with the actual conditions of different employees; the wearer's action mapping sequence is inferred based on the action probability distribution, so that the obtained action mapping sequence is more accurate.

[0129] In some embodiments of the present specification, the motion mapping sequence of the wearer is determined based on the wearer's busyness, and the motion mapping sequence of the wearer can be accurately determined at different busyness levels.

[0130] Figure 5 FIG. 1 is an exemplary schematic diagram of determining an action evaluation result according to some embodiments of this specification. Figure 5 As shown, the microprocessor can determine the action mapping subsequence 510 from the action mapping sequence 490; determine the action evaluation result 570 based on the action mapping subsequence 510, the customer flow data 520 of the preset time period, and the stage order volume 530 of the preset time period. For more information about the action mapping sequence and the action evaluation result, see Figure 3 Related instructions.

[0131] In the process of store management, there is a situation where it is necessary to judge whether the operations in a certain period of time or for a few orders meet the requirements. At this time, it is necessary to intercept the data of a certain period of time in the action mapping sequence and analyze this data as an action mapping subsequence.

[0132] An action mapping subsequence is a sequence intercepted from an action mapping sequence. The action mapping subsequence includes actions corresponding to a preset time period in the action mapping sequence.

[0133] In some embodiments, the microprocessor may extract a sequence of a preset time period from the action mapping sequence to obtain an action mapping subsequence.

[0134] The action mapping sequence includes the actions of the wearer for multiple orders in the past period of time. If only the actions of a part of the orders need to be evaluated, the required action mapping subsequence can be intercepted from the action mapping sequence, and only the action mapping subsequence is processed. For example, the microprocessor can obtain user input to determine the order that needs to be evaluated. Among them, the user input can include a specific order number, and can also include the status information or time information of the order. For example, the user input can be "obtain the action mapping subsequence corresponding to the order with order number XXX", or "obtain the action mapping subsequence corresponding to the order with order status completed", or "obtain the action mapping subsequence corresponding to the order with order processing completion time between time A and time B". The user input can also be other content or form, which can be set according to actual needs.

[0135] In some embodiments, the microprocessor can determine the action mapping subsequence corresponding to the order from the action mapping sequence based on the completion time and standard processing time of a certain order. For example, the microprocessor can determine the element corresponding to the completion time from the action mapping sequence based on the completion time of the order, and determine the element as the end element of the action mapping subsequence, determine the estimated start time of the order based on the standard processing time and completion time of the order, determine the element corresponding to the estimated start time from the action mapping sequence, and determine the element as the starting element of the action mapping subsequence, and determine the starting element and the ending element and the elements between them as the action mapping subsequence. Among them, the standard processing time refers to the time from the start of order processing to the completion of order processing, which can be determined based on the average processing time of similar orders in historical data. For example, the standard processing time can be determined based on a preset multiple of the aforementioned average processing time, and the preset multiple is a value not less than 1, which can be set based on actual needs.

[0136] In some embodiments, the duration of the preset time period can be set based on actual needs. If you want to evaluate the overall execution effect of the action, the duration of the preset time period is longer, and if you want the evaluation result to be more targeted, the duration of the preset time period is shorter.

[0137] In some embodiments, the duration of the preset time period is related to the frequency of complaints against the wearer. The higher the frequency of complaints against the wearer, the longer the duration of the preset time period.

[0138] The microprocessor can obtain the frequency of complaints against the wearer by exchanging data with the store's backend management system.

[0139] If the wearer has a high frequency of complaints, it means that the customer is not satisfied with the wearer's actions. By increasing the length of the preset time period, the wearer's actions can be evaluated more comprehensively, thus avoiding falling into a local trap and resulting in an incomplete evaluation of the person.

[0140] In some embodiments, the microprocessor may determine the action evaluation result based on the action mapping subsequence 510, the customer flow data 520 of the preset time period, and the stage order quantity 530 of the preset time period.

[0141] Exemplarily, the microprocessor can determine the similarity threshold based on the passenger flow data of the preset time period and the stage order volume of the preset time period; calculate the similarity between the action mapping subsequence and multiple standard action mapping sequences in the standard sequence library, and determine the action evaluation result based on the maximum value of the multiple similarities. For example, if the maximum value of the multiple similarities is greater than the aforementioned similarity threshold, the evaluation result is qualified, otherwise, the evaluation result is unqualified. For an explanation of the standard sequence library, please refer to this specification. Figure 3 Related description in .

[0142] In some embodiments, the similarity threshold is negatively correlated with the customer flow data of a preset time period and the stage order volume of the preset time period.

[0143] In some embodiments, the microprocessor can obtain the customer flow data and the stage order volume of the preset time period based on a variety of methods. For example, the microprocessor can obtain the customer flow data of the preset time period by obtaining user input, infrared sensing device technology, video analysis technology, AI scene recognition, etc., and obtain the stage order volume of the preset time period by reading the data of the store system such as the backend system and the cash register system, and performing statistics on the data.

[0144] In some embodiments, the microprocessor can determine the action standard 540 based on the action mapping subsequence 510; determine the action execution degree 550 based on the customer flow data 520, the stage order quantity 530 and the action mapping subsequence 510; and determine the action evaluation result 570 according to the preset rules 560 based on the action standard 540 and the action execution degree 550.

[0145] The standard of action reflects the degree of standardization of the action. The higher the standard of action, the more standardized the action.

[0146] In some embodiments, the microprocessor may determine the action execution standard corresponding to at least one action in the action mapping subsequence based on the action mapping subsequence; and determine the action standard degree based on at least one action in the action mapping subsequence and its corresponding action execution standard. The action execution standard refers to the standardized execution process of the action, which may include time requirements, execution area requirements, standardized action requirements, etc. Different work contents correspond to different action execution standards, and the action execution standards can be preset.

[0147] For example, the action execution standards corresponding to task A may include {(A1, 3, M), (A2, 5, N), (A3, 1, M)}, where element (A1, 3, M) means that when completing task A, action A1 must be performed in area M for at least more than 3 minutes, and other elements can be understood similarly.

[0148] Exemplarily, the microprocessor can compare the multiple actions in the action mapping subsequence and their corresponding occurrence areas with the action execution standard, count the number of actions that are completely consistent with the action execution standard, and determine the ratio of the number of completely consistent actions to the total number of standard actions as the action standard of the action mapping subsequence. For example, if the action execution standard corresponding to task A is {(A1, 3, M), (A2, 5, N), (A3, 1, M)}, and actions A1 and A2 in the action mapping subsequence are completely consistent with the action execution standard, then the action standard corresponding to the action mapping subsequence is 2 / 3.

[0149] The action execution degree reflects the completion of the action. The greater the action execution degree, the better the action completion. For example, the higher the action execution degree corresponding to the action mapping subsequence, the more standard actions are completed in the action mapping subsequence according to the standard order.

[0150] In some embodiments, the microprocessor can calculate the similarity between the action mapping subsequence and each standard action mapping sequence in the standard sequence library, sort them in descending order of similarity, use a preset number of standard action mapping sequences with the highest ranking as target sequences, and determine the average value of the similarity corresponding to the target sequences as the action execution degree of the action mapping subsequence. In some embodiments, the standard sequence library can include at least one standard action mapping sequence, and each standard action mapping sequence can be determined based on user input.

[0151] In some embodiments, the preset number is related to the complexity of the action mapping subsequence. The more action types and actions there are in the action mapping subsequence, the higher the complexity is and the larger the preset number is.

[0152] The more complex the action mapping subsequence is, the less clear its meaning may be, or the more complex the task to be performed is, the more complex its standard degree evaluation may be, and more data is needed to evaluate its action execution degree.

[0153] In some embodiments, the microprocessor may determine the action evaluation result according to preset rules based on the action standard and the action execution degree.

[0154] In some embodiments, the preset rule is: determine the action evaluation value based on the weighted sum of the action standard and the action execution degree, if the action evaluation value is less than the evaluation threshold, the action evaluation result is unqualified, otherwise, the action evaluation result is qualified. The weights corresponding to the action standard and the action execution degree in the weighted sum can be set based on prior experience.

[0155] The evaluation threshold is a threshold used to determine whether an action is qualified. In some embodiments, the microprocessor can determine the evaluation threshold based on historical evaluation data. For example, the microprocessor can count the unqualified action mapping sequences actually monitored in the historical data, and calculate the historical action evaluation values ​​corresponding to the unqualified action mapping sequences, and determine the largest historical action evaluation value as the evaluation threshold.

[0156] In some embodiments, the evaluation threshold is also related to the store rating and the frequency of complaints against the wearer.

[0157] The store rating refers to the grade of the evaluated store. In some embodiments, the store rating can be obtained based on user input, or by calling data in other databases (such as calling data in a public business information database, etc.).

[0158] In some embodiments, the evaluation threshold is positively correlated with both the store rating and the wearer's complaint frequency.

[0159] The higher the store rating, the higher the industry standards the store needs to meet, and the higher the requirements for action evaluation. If the wearer receives complaints more frequently, the wearer needs to be subject to strict regulations and the requirements for action evaluation will be higher.

[0160] Some embodiments of the present specification can evaluate the action more comprehensively by evaluating the action mapping subsequence from two aspects: action standardization and action execution, thereby obtaining a more accurate action evaluation result.

[0161] The more people flow and the more orders there are, the busier the employees will be. It is understandable that when they are busy, they will adaptively adjust some processing steps to meet efficiency requirements. Therefore, the standards can be appropriately lowered when they are busy to make the action evaluation results more humane.

[0162] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0163] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0164] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0165] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0166] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0167] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A user action recognition system based on a wearable terminal, characterized in that: It includes motion monitoring equipment, data acquisition components, positioning components, communication components, alarm components and microprocessors; The motion monitoring device is configured to obtain motion data of the wearer; The data acquisition component is configured to collect ambient audio; The positioning component is configured to obtain the location data of the wearer and determine the current area corresponding to the wearer; The communication component is configured to realize data interaction among the motion monitoring device, the data acquisition component, the positioning component, the alarm component and the microprocessor; The alarm component is configured to send a reminder to the wearer; The microprocessor is configured to: Determining a motion mapping sequence of the wearer based on the motion data, the ambient audio, and the current area; Determining a motion evaluation result of the wearer based on the motion mapping sequence; In response to the action evaluation result not meeting a preset condition, the reminder is sent to the wearer via the alarm component.

2. The system according to claim 1, characterized in that The microprocessor is further configured to: Determine a load index based on the ambient audio and the current area, wherein the load index reflects the busyness of the wearer in the current area; Based on the load index and the motion data, the motion mapping sequence is determined.

3. The system according to claim 2, characterized in that The processor is further configured to: determining an action probability distribution based on wearer information of the wearer; The action mapping sequence is determined based on the motion data, the action probability distribution and the load index.

4. The system according to claim 1, characterized in that The microprocessor is further configured to: Determining an action mapping subsequence from the action mapping sequence, the action mapping subsequence including actions corresponding to a preset time period in the action mapping sequence; The action evaluation result is determined based on the action mapping subsequence, the customer flow data of the preset time period, and the stage order quantity of the preset time period.

5. The system according to claim 4, characterized in that The microprocessor is further configured to: Determining an action standard degree based on the action mapping subsequence; Determining an action execution degree based on the customer flow data, the stage order quantity and the action mapping subsequence; Based on the action standard degree and the action execution degree, the action evaluation result is determined according to a preset rule, and the preset rule is related to an evaluation threshold.

6. A user action recognition method based on a wearable terminal, characterized in that: The method is executed by a microprocessor and includes: Determine the motion mapping sequence of the wearer based on the motion data ambient audio and the current area; Determining a motion evaluation result of the wearer based on the motion mapping sequence; In response to the action evaluation result not meeting a preset condition, the reminder is sent to the wearer via the alarm component, and the preset condition is related to whether the action evaluation result is qualified.

7. The method according to claim 6, characterized in that The step of determining the motion mapping sequence of the wearer based on the motion data, the ambient audio, and the current area comprises: Determine a load index based on the ambient audio and the current area, wherein the load index reflects the busyness of the wearer in the current area; Based on the load index and the motion data, the motion mapping sequence is determined.

8. The method according to claim 7, characterized in that The step of determining the action mapping sequence based on the load index and the motion data comprises: determining an action probability distribution based on wearer information of the wearer; The action mapping sequence is determined based on the motion data, the action probability distribution and the load index.

9. The method according to claim 6, characterized in that The step of determining the motion evaluation result of the wearer based on the motion mapping sequence includes: Determining an action mapping subsequence from the action mapping sequence, the action mapping subsequence including actions corresponding to a preset time period in the action mapping sequence; The action evaluation result is determined based on the action mapping subsequence, the customer flow data of the preset time period, and the stage order quantity of the preset time period.

10. The method according to claim 9, characterized in that The determining the action evaluation result based on the action mapping subsequence, the passenger flow data in the preset time period, and the stage order quantity in the preset time period includes: Determining an action standard degree based on the action mapping subsequence; Determining an action execution degree based on the customer flow data, the stage order quantity and the action mapping subsequence; Based on the action standard degree and the action execution degree, the action evaluation result is determined according to a preset rule, and the preset rule is related to an evaluation threshold.

Images