Method, system, device and medium for predicting age group or gender of a user

The age group and gender are predicted through user gesture information, and the gesture data is processed using machine learning models, which solves the problems of high equipment costs, difficulty in privacy protection and high power consumption in the prior art, and achieves the prediction effect of low-cost and low power consumption.

CN115081334BActive Publication Date: 2024-11-26ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210768050.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-30
Publication Date
2024-11-26
Estimated Expiration
2042-06-30

AI Technical Summary

Technical Problem

When predicting the age or gender of a user, the prior art requires collecting and processing a large amount of image, sound or action data, resulting in high equipment costs, difficulty in protecting users' privacy, high processing complexity and increased power consumption.

Method used

Generate gesture information through user's touch panel gesture input, and use machine learning models, including trajectory modules, timing modules and category modules, to predict based on features such as gesture event type, position, duration, and pressing pressure to reduce dependence on additional sensors.

Benefits of technology

It realizes low-cost, low-power user age and gender prediction, protects user privacy, reduces equipment costs, and improves system performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115081334B_ABST
    Figure CN115081334B_ABST
Patent Text Reader

Abstract

A method for predicting the age group or gender of a user is disclosed, comprising: receiving a gesture input of the user on a touch panel; generating gesture information based on the gesture input; and using a machine learning model to predict at least one of the age group and the gender of the user based on the gesture information. A system, an apparatus and a medium for predicting the age group or gender of a user are also disclosed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to user information prediction, and in particular to a method, system, device and medium for predicting a user's age group or gender. Background Art

[0002] The user's age group and gender are important user information. However, in many cases, the user may not provide age group or gender information. In this case, the user's age group or gender information needs to be predicted.

[0003] Current models usually use user images, sounds, or motion information to perform predictions. However, such models may require the collection and processing of more user data (sometimes even requiring the use of additional sensor modules), which has adverse effects on user privacy protection, device cost and complexity, device processing power, and power consumption.

[0004] Therefore, there is a need for a solution that can predict the age group and / or gender of a user efficiently and at low cost. Summary of the invention

[0005] In order to overcome the defects of the prior art, one or more embodiments of the present specification utilize the user's gesture information to achieve a prediction solution for the user's age group or gender that reduces the amount of data collection and does not require additional sensors.

[0006] One or more embodiments of the present specification achieve the above-mentioned purpose through the following technical solutions.

[0007] In one aspect, a method for predicting a user's age group or gender is provided, comprising: receiving gesture input from a user on a touch panel on a user device; generating gesture information based on the user's gesture input, the gesture information comprising two or more of gesture event type, gesture position, gesture duration, average pressing pressure, average pressing area, gesture timestamp, and associated business identifier; and using a machine learning model to predict at least one of the user's age group and gender based on the gesture information.

[0008] Preferably, the machine learning model utilizes one or more of the trajectory features, timing features and category features of the gesture input to predict at least one of the age group and gender of the user.

[0009] Preferably, the machine learning model includes a trajectory module, and the trajectory module is used to generate a trajectory feature code of the gesture input based on the gesture event type, the gesture starting point coordinates and / or the gesture ending point coordinates.

[0010] Preferably, the trajectory module is used to: convert the gesture event type, gesture starting point coordinates and / or gesture ending point coordinates into a trajectory image; and generate a trajectory feature code of the gesture input based on the trajectory image using a CNN-based model.

[0011] Preferably, the machine learning model includes a timing module, which is used to generate a timing feature encoding of the gesture input based on gesture duration, average pressing pressure, average pressing area, and / or gesture timestamp.

[0012] Preferably, the timing module is used to: generate sampling information using a convolution algorithm based on the gesture duration, average pressing pressure, average pressing area, and / or gesture timestamp; and input the sampling information into an attention mechanism to generate a timing feature encoding of the gesture input.

[0013] Preferably, the machine learning model includes a category module, which is used to: embed the associated business identifier of the gesture input into a word to generate a word embedding representation; generate one or more artificial features based on the gesture information; splice the artificial features and the word embedding representation, and input the spliced ​​output into a wide neural network model to generate the category feature encoding.

[0014] Preferably, the method further comprises concatenating feature codes of two or more of the trajectory features, time sequence features and category features to generate a gesture feature code.

[0015] Preferably, the method further comprises using a multi-gated hybrid expert module of the machine learning model to predict both the age group and gender of the user based on the gesture information.

[0016] Preferably, the method further comprises: using an XGBoost model to preliminarily divide the age groups of users.

[0017] Preferably, the method further comprises: predicting at least one of the age group and gender of the user based on a combination of the gesture information of the user and one or more of the body shape information, movement information and sound information of the user.

[0018] Preferably, the method further comprises: determining an identity authentication method for the user based on at least one of the predicted age group and gender of the user.

[0019] Preferably, the method further comprises generating a recommendation for the user based on at least one of the predicted age group and gender of the user.

[0020] On the other hand, a system for predicting a user's age group or gender is provided, comprising: a gesture input receiving module for receiving gesture input from a user on a touch panel on a user device; a gesture information generating module for generating gesture information based on the user's gesture input, the gesture information comprising two or more of gesture event type, gesture position, gesture duration, average pressing pressure, average pressing area, gesture timestamp, and associated business identifier; and a prediction module for predicting at least one of the user's age group and gender based on the gesture information using a machine learning model.

[0021] Preferably, the machine learning model includes one or more of a trajectory module, a timing module, and a category module.

[0022] In yet another aspect, a device for predicting an age group or gender of a user is provided, comprising: a memory; and a processor, wherein the processor is configured to execute the above method.

[0023] In yet another aspect, a computer-readable storage medium storing instructions is provided, which instructions, when executed by a computer, cause the computer to perform the above method.

[0024] Compared with the prior art, one or more embodiments of the present specification can achieve one or more of the following technical effects:

[0025] Does not rely on additional sensors and / or collection of additional information;

[0026] Better protect user privacy;

[0027] Makes user equipment costs lower;

[0028] is more widely applicable; and / or

[0029] It reduces processor occupancy, improves system performance, and reduces power consumption.

[0030] It should be understood that the technical effects achieved by one or more embodiments of this specification are not limited to the above items, and one or more embodiments of this specification are not required to achieve all of the above technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above invention content and the following detailed description will be better understood when read in conjunction with the accompanying drawings. It should be noted that the accompanying drawings are only examples of the claimed invention. In the accompanying drawings, the same reference numerals represent the same or similar elements.

[0032] Figure 1A An exemplary schematic diagram showing the amount of data of a user according to an embodiment of the present specification.

[0033] Figure 1B The machine learning model is used to Figure 1A The recall rate of the preliminary age group classification of users in .

[0034] Figure 2 An example of a data structure of gesture information according to an embodiment of the present specification is shown.

[0035] Figure 3 A block diagram showing the architecture of a machine learning model for predicting a user's age group and / or gender according to a preferred embodiment of the present specification.

[0036] Figure 4 A schematic block diagram showing an example structure of a trajectory module according to an embodiment of the present specification.

[0037] Figure 5 An example schematic diagram showing how to convert trajectory information into a trajectory image according to an embodiment of the present specification.

[0038] Figure 6 A schematic block diagram showing an example structure of a timing module according to an embodiment of the present specification.

[0039] Figure 7 A schematic block diagram showing an example structure of a category module according to an embodiment of the present specification.

[0040] Figure 8 A schematic block diagram showing an example structure of a classification module according to an embodiment of the present specification.

[0041] Fig. 9 A flowchart showing an example method for predicting the age group or gender of a user according to an embodiment of the present specification.

[0042] Fig.10 A schematic block diagram showing an example system for predicting the age group or gender of a user according to an embodiment of the present specification.

[0043] Fig.11 A schematic block diagram showing an apparatus for implementing a method according to one or more embodiments of the present specification. DETAILED DESCRIPTION

[0044] The contents of the following specific implementations are sufficient to enable any technical person skilled in the art to understand the technical contents of one or more embodiments of this specification and implement them accordingly, and based on the description, claims and drawings disclosed in this specification, those skilled in the art can easily understand the purposes and advantages related to one or more embodiments of this specification.

[0045] The user's age group and gender are important user information. However, in many cases, the user may not provide age group or gender information. In this case, the user's age group or gender information needs to be predicted.

[0046] In order to predict the user's model, machine learning models have been increasingly used in recent years. Machine learning models will use one or more types of data (such as user data) to predict the user's age group or gender.

[0047] In one approach, a machine learning model can use a user's body shape data to predict the user's age group or gender.

[0048] The user's posture data may include, for example, the user's hand motion information, hip motion information, arm motion information, wrist motion information, chest motion information, etc. when walking. It is conceivable that the postures of users of different age groups (such as the elderly, young people, and children) may be different, and the postures of users of different genders may also be different. Therefore, by training the machine learning model with the posture data of multiple users, a machine learning model that predicts the user's age group or gender based on the posture data can be obtained.

[0049] However, the user's body posture data usually needs to be collected through sensors such as accelerometers and gyroscopes. Therefore, to collect the user's body posture data, not only the user's device needs to have the above sensors, but also the user needs to authorize the use of the above sensors to collect information, and the processor processing power and power consumption are required to perform data collection and process the collected data.

[0050] In another scenario, a machine learning model can use a user's voice data to predict the user's age group or gender.

[0051] The user's voice data may include, for example, the user's speech data. Voice data is a common data type that can be used to profile a user. In one example, the prediction accuracy can be improved by guiding the user to say a specific sentence. In an alternative example, the prediction can be performed by randomly collecting the user's voice. It can be understood that the voices of users of different age groups and different genders have their own characteristics, so the user's age group and / or gender can be predicted through voice data. Various machine learning models can be used to perform predictions, such as deep learning voice networks.

[0052] Similarly, the user's voice data usually needs to be collected by a dedicated element such as a microphone (e.g., a built-in microphone in a user device such as a smartphone). Again, this requires the user device to have a microphone, user authorization to use the microphone to collect sound, processor processing power, and power consumption to perform data collection and process the collected data.

[0053] In yet another approach, a machine learning model may use the user's motion data to predict the user's age group or gender.

[0054] The user's motion data may include, for example, various conceivable motion data such as the user's air writing motion, eye movement, shaking motion, free motion, etc. The user's motion data can usually be collected by an image acquisition device (such as a camera), which is different from the body posture data collected by an accelerometer and a gyroscope as mentioned above. It can be understood that the motions of users of different age groups and different genders have their own characteristics (for example, in terms of motion frequency, motion amplitude, motion habits, etc.), so the user's age group and / or gender can be predicted through motion data. Various machine learning models can be used to perform predictions, such as various image processing models.

[0055] Similarly, the user's motion data usually needs to be collected by a dedicated element such as an image acquisition device (e.g., a camera). Similarly, this requires the user's device to have an image acquisition device, requires user authorization to use the device to capture the user's image, requires processor processing power and consumes power to perform data acquisition and process the collected data.

[0056] It can be seen from the above that special sensors or data acquisition elements are usually required to collect corresponding data, which has adverse effects on device cost, user privacy, device performance and power consumption.

[0057] In response to the above problems, the inventor creatively realized that users of different age groups and genders may also have different gesture behavior habits. For example, the operation frequency of the elderly is usually slower; the pressing pressure of children and women is usually smaller; and the services clicked by users of different age groups and genders are often different.

[0058] Therefore, the embodiments of this specification provide a solution for predicting the user's age and / or gender using gesture data. This solution not only proposes an idea that has not been found in the prior art for predicting the user's age and / or gender, but also solves the above problems compared to the prior solutions.

[0059] In order to predict the age of users, we can first divide them into age groups. There are many ways to divide age groups. For example, the following age group division schemes can be used:

[0060] Plan 1: 6-10 years old, 10-20 years old, 20-30 years old, 30-40 years old, 40-50 years old, 50-60 years old, and over 60 years old;

[0061] Option 2: 6-20 years old, 20-50 years old, and over 50 years old;

[0062] Plan 3: 6-40 years old, 40-60 years old, and over 60 years old;

[0063] etc.

[0064] It can be appreciated that different age group division schemes may also affect the performance of the machine learning model in predicting age groups. For example, if an age range that does not have significant uniform characteristics is divided into one age group, it may have an adverse effect on the prediction of the age group.

[0065] In general, age groups can be divided based on experience. For example, the age group of the user can be set by the developer and / or the user of the prediction system based on their experience or historical data. However, dividing age groups based on subjective experience may not achieve the most reasonable age group division.

[0066] In the preferred example of this specification, the user's age groups can be first divided using a machine learning model, so that age ranges with more uniform and prominent features can be divided into the same age group, so as to improve system performance in subsequent prediction processes.

[0067] Figure 1A An example schematic diagram showing the amount of data of a user according to an embodiment of the present specification. Figure 1A In the example, the horizontal axis represents age (in years), and the vertical axis represents the amount of data of users of the same age, which is calculated as the number of data points (number of items), indicating the number of available data items for the user. The amount of data may come from a test data set, for example.

[0068] like Figure 1A As shown, due to the availability of data, the amount of data for users under 6 years old and over 65 years old is less, while the amount of data for users in the middle range (especially 20-40 years old) is larger.

[0069] Figure 1B The machine learning model is used to Figure 1A The recall rate of the preliminary age group classification of users in . Figure 1B In the figure, the horizontal axis represents age (in years), and the vertical axis represents the recall rate of the age group predicted by the machine learning model.

[0070] The machine learning model can be any known machine learning model, such as an XGBoost model. The recall rate is the recall rate when the age groups are divided into 6-16 years old (teenagers), 17-50 years old (adults) and 51-100 years old (elderly).

[0071] like Figure 1BAs shown in the figure, the recall rate of 17-year-olds is very high while the recall rate of 50-year-olds is very low. It can be seen that there are obvious differences between users of different age groups. Based on the above results, it is more appropriate to divide the age groups of the training data into 6-16 years old (teenagers), 17-50 years old (adults), and 51-100 years old (elderly). By making a preliminary division of age groups based on the performance of recall rate, the system performance in subsequent age group prediction can be improved.

[0072] As described above, in the embodiment of the present specification, a machine learning model is proposed to predict at least one of the age group and gender of the user based on gesture information. The gesture information can be generated based on the gesture input of the user on the touch panel of the user device.

[0073] The user device may be any computing device with a touch panel, including but not limited to: a smart phone, a tablet computer, a notebook computer, a fixed computer, an embedded computer, and various types of Internet of Things (IoT) devices, etc. The touch panel may receive a user's touch input and convert the user's touch input into information such as coordinate points.

[0074] Preferably, the touch panel is a touch screen, that is, a device having both a touch function and a display function. Alternatively, the touch panel may not have a display function. The following will be described using a touch screen as an example.

[0075] Examples of gesture input may include, but are not limited to, touch gestures, click gestures, page turning gestures, sliding gestures, sweeping gestures, key gestures, etc.

[0076] Based on the number of input points, gesture input can be divided into single-point gesture input and multi-point gesture input (such as multi-point touch input, multi-point sliding input, etc.). Preferably, the gesture input used in the embodiments of this specification is single-point gesture input. In some cases, the gesture input used in the embodiments of this specification may also include multi-point gesture input.

[0077] Based on the event type, gesture input can be divided into click input and sliding input, where click input is usually only related to static coordinate points (in the case of multi-point input, it may be multiple static coordinate points), while sliding input may include dynamic coordinate points that constitute the action path of the input gesture, which usually has a starting point and an end point.

[0078] See also Figure 2 , which shows an example of the data structure of gesture information according to an embodiment of the present specification.

[0079] like Figure 2As shown, in the example of this specification, the gesture information may include event type, starting point coordinates (which may be represented by starting point abscissa and starting point ordinate), end point coordinates (which may be represented by end point abscissa and end point ordinate), gesture duration, average pressing pressure, average pressing area, gesture timestamp, associated service identifier (for example, see Figure 2 The gesture information may also include information derived from the above information. The gesture information may also include any other information associated with a gesture on the touch screen of the user device that can be conceived by those skilled in the art.

[0080] As described above, the event type may indicate the type of the event, such as a click event and a slide event.

[0081] For a sliding event, the starting point coordinates are the coordinates of the starting point of the sliding path. For a clicking event, the starting point coordinates are the coordinates of the clicked point.

[0082] For a sliding type event, the end point coordinates are the coordinates of the end point of the sliding path. For a click type event, the end point coordinates may not be included (the corresponding value may be set to null), or the end point coordinates may be the same as the start point coordinates.

[0083] The gesture duration may be the duration from when the finger begins to contact the touch screen to when the finger leaves the touch screen.

[0084] The average pressing pressure may be an average value of the pressing pressure of the finger on the touch screen during the gesture duration.

[0085] The average pressed area may be an average value of the contact area of ​​the finger with the touch screen during the gesture duration.

[0086] The gesture timestamp may be a timestamp when a finger contacts the touch screen.

[0087] The associated service identifier may indicate the service that the user is interacting with through the gesture input, for example, it may be the identifier of the service displayed on the touch screen when the gesture just touches the touch screen or during the duration of the gesture. For an application, the identifier may be an application ID (e.g., APP ID). For a page (e.g., a page within an application, a web page, etc.), the identifier may be a page ID or a page URI, etc.

[0088] In a preferred example, the gesture information may be expressed as follows: Figure 2The data structure shown in FIG. 1 may be an MxN matrix, where M is the number of features and N is the number of data points. For example, M may be 10 (i.e., the 10 features of the gesture information described above, as shown in indexes 0-9), and N may be 200, i.e., 200 interactions of the user with the touch screen (e.g., the most recent 200 interactions). It is understood that more or fewer features may be selected based on the specific situation. In addition, each matrix may use more or fewer data points.

[0089] Any other data structure conceivable by those skilled in the art may also be employed.

[0090] See also Figure 3 , which shows a block diagram of the architecture of a machine learning model 300 for predicting a user's age group and / or gender according to a preferred embodiment of the present specification.

[0091] like Figure 3 As shown, in a preferred embodiment, the machine learning model 300 may include a module for encoding one or more types of gesture features (which may be referred to as a "gesture feature module"), such as one or more of a trajectory module 302, a timing module 304, and a category module 306. The trajectory module 302, the timing module 304, and the category module 306 may receive the gesture information 301, and generate feature encodings of trajectory features, timing features, and category features based on the gesture information, respectively.

[0092] It should be appreciated that, although all three modules are described herein and shown in the drawings, it should be appreciated that the age group / gender prediction of the embodiments of the present specification can also be achieved by only one or any two of the modules.

[0093] When there are multiple gesture feature modules, the machine learning model 300 may include a splicing module 308. The feature codes output by the multiple gesture feature modules may be spliced ​​into a combined gesture feature code by the splicing module 308. The combined gesture feature code may be used to predict one or both of the age group and the gender. One or more of the trajectory module 302, the timing module 304, the category module 306 and the splicing module 308 together constitute a combined feature code module 310, which is used to generate a combined gesture feature code based on the gesture information for output to the classification module 312.

[0094] When there is only one gesture feature module (i.e., there is only one of the trajectory module 302, the timing module 304 and the category module 306), the machine learning model 300 may not include the splicing module 308. At this time, the output of the one gesture feature module is the gesture feature code input to the classification module 312.

[0095] The generated gesture feature code is input to the classification module 312 to generate a prediction of at least one of the age group 313 and gender 315 of the user based on the gesture feature code.

[0096] When predicting both the age group and the gender, the classification module 312 may be a multi-task classification module. When predicting only one of the age group and the gender, the classification module 312 may be a single-task classification module.

[0097] In the following, we will discuss each Figure 2 The modules shown in are introduced.

[0098] See also Figure 4 , which is a schematic block diagram showing an example structure of a trajectory module 400 according to an embodiment of this specification.

[0099] like Figure 4 As shown, the trajectory module 400 can receive information related to the trajectory of the gesture, that is, the trajectory information 401 of the gesture. The trajectory information may include, for example, the gesture event type, the coordinates of the starting point of the gesture (including the horizontal coordinate and the vertical coordinate) and / or the coordinates of the end point of the gesture (including the horizontal coordinate and the vertical coordinate, and the coordinates of the end point of the gesture may not be included for a click type event). It can be understood that the trajectory information may include more or less information.

[0100] In one example, the following may be used: Figure 3 Features 0 to 4 in the data structure shown form a 200x5 trajectory information matrix.

[0101] The trajectory module 400 may include an image conversion module 402 and an image processing module 404 .

[0102] In order to perform processing, in the embodiment of the present specification, the trajectory information is converted into a trajectory image by the image conversion module 402. Specifically, the touch panel can be converted into a canvas of a specified size, such as 100x50. Subsequently, the trajectory information can be converted into a trajectory image in the canvas.

[0103] Figure 5 An example schematic diagram of converting trajectory information into a trajectory image according to an embodiment of this specification is shown.

[0104] For example, Figure 5 As shown, for a sliding type event, the trajectory image may be a curve segment on the canvas, such as a straight line segment from the starting point (x1, y1) to the end point (x2, y2). For example, in a canvas representing a touch panel, points on the curve segment may be assigned a value of 1, while points not on the curve segment may be assigned a value of 0, thereby obtaining a 100x50 matrix. Therefore, the original 200x5 trajectory information matrix may be converted into a 200x100x50 trajectory image matrix.

[0105] For click type events, the track image can be a point on the canvas ( Figure 5 At this time, only one element (i.e., the point corresponding to the starting point coordinates) in the 100x50 matrix may be assigned a value of 1, while the other elements are assigned a value of 0.

[0106] Subsequently, the trajectory image matrix may be input to the image processing module 404 for processing. Preferably, the image processing module may be a convolutional neural network (CNN) model, or any other CNN-based model. Any other image processing model that can be conceived by those skilled in the art may also be used.

[0107] Through the image processing model, the encoding of the trajectory feature of the gesture input, that is, the trajectory feature encoding, can be generated.

[0108] See also Figure 6 , which is a schematic block diagram showing an example structure of a timing module 600 according to an embodiment of this specification.

[0109] like Figure 6 As shown, the timing module 600 can receive information related to the timing or time of the gesture, that is, the timing information of the gesture. The timing information may include, for example, the duration of the gesture, the average pressing pressure, the average pressing area, and the timestamp. It is understood that the timing information may include more or less information.

[0110] In one example, the following may be used: Figure 4 The features indexed from 5 to 8 in the data structure shown form a 200x4 time series information matrix.

[0111] like Figure 6 As shown, the timing module 600 may include a convolution module 602 and an attention mechanism module 604. The convolution module 602 may perform a convolution algorithm based on the timing information to generate sampling information. For example, through the convolution module, a 200x64 sampling information matrix may be obtained based on a 200x4 timing information matrix.

[0112] The convolution module 602 may perform convolution on the time series information 601 matrix. Preferably, a convolutional neural network with a 1x1 convolution kernel may be used to process the time series information matrix. Other convolution kernels may also be used to perform convolution. Through convolution, sampling information may be generated.

[0113] The attention mechanism module 604 may generate processing for the sampled information to generate temporal feature encoding of the gesture information.

[0114] like Figure 6As shown, the attention mechanism module 604 can use the sampling information and the corresponding position code 603 to use multiple attention mechanism layers 606 for processing, and each attention mechanism layer 606 may include, for example, a single-head or multi-head attention mechanism unit, a summation and normalization unit, and a feedforward network. The specific implementation of the attention mechanism module 604 (including the position code 603) can refer to the Transformer model or its variants. For example, refer to the paper "Attention is all you need" published by Ashish Vaswani et al. in 2017 (see https: / / arxiv.org / abs / 1706.03762).

[0115] See also Figure 7 , which is a schematic block diagram showing an example structure of a category module 700 according to an embodiment of the present specification.

[0116] like Figure 7 As shown, the category module 700 can receive information related to the category of the gesture, that is, the category information 701 of the gesture. In the embodiment of this specification, the category information refers to the associated business identifier of the gesture input, such as an APP ID. The category information may include more or less information. It can be imagined that the patterns of gesture behaviors of users of different age groups and / or genders in different businesses may be different, so adding category information such as business identifiers helps to identify the age group and / or gender of the user.

[0117] In one example, the following may be used: Figure 4 The feature with index 9 in the data structure shown forms a 200x1 time series information matrix.

[0118] In addition, the category module 700 may include a word embedding module 702, which may generate a word embedding representation of the category information (e.g., the associated business identifier). Since the category information such as the business identifier itself may not have obvious meaning, a simple word embedding algorithm may be used to generate its word embedding representation.

[0119] The category module 700 may further include a manual feature generation module 704. The manual feature generation module may generate a specified manual feature based on the gesture information 703. The gesture information may be, for example, the one described above. Figure 2 Manual information shown.

[0120] Examples of manual features may include, but are not limited to: gesture sliding speed, pressing pressure, gesture input frequency, and the like.

[0121] The gesture sliding speed can be obtained by dividing the sliding distance (for example, the straight-line distance from the start point to the end point of the gesture) by the gesture duration. The gesture sliding speed may not be calculated for a click type event.

[0122] The pressing pressure can be obtained by dividing the pressing pressure of the gesture by the pressing area.

[0123] The gesture input frequency can be obtained by counting the gesture operations within a unit time.

[0124] In a specific application, a statistical value may be calculated for each artificial feature as the artificial feature actually used. For example, the maximum value, minimum value, median value, average value, variance, etc. of each artificial feature may be calculated.

[0125] Any other artificial features that can be conceived by those skilled in the art may be employed.

[0126] like Figure 7 As shown, the category module 700 may further include a concatenation module 706 and a wide neural network module 708. The concatenation module 706 may generate a concatenated category feature representation based on the artificial features and the word embedding representation of the associated business identifier.

[0127] Subsequently, the concatenated category feature representation is input into the wide neural network module 708 to generate the category feature encoding 709. Preferably, the wide neural network module 708 may include a fully connected layer. The inventors found that the wide neural network performs well for category features. The specific implementation of the wide neural network is not described here.

[0128] As mentioned above, if Figure 3 As shown, the trajectory feature code generated by the trajectory module 302, the timing feature code generated by the timing module 304, and the category feature code generated by the category module 306 can be spliced ​​by the splicing module 308 to generate a spliced ​​code as the final gesture feature code.

[0129] It should be understood that although Figure 3 All three modules are shown in FIG. 1 , but the machine learning model may include any one or any two modules. When only two modules are included, the concatenation module may only concatenate feature codes from the two modules. When only one module is included, the concatenation module may not be included.

[0130] See also Figure 8 , which is a schematic block diagram showing an example structure of a classification module 800 according to an embodiment of this specification.

[0131] As described above, when the age group and the gender are predicted simultaneously, the classification module may be a multi-task classification module. When only one of the age group and the gender is predicted, the classification module may be a single-task classification module.

[0132] When the classification module is a multi-task classification module, the multi-task classification module may adopt an MMoE (Multi-Gated Mixture of Experts) model.

[0133] Since there is also a correlation between age group and gender in terms of gestures (for example, the feature differences between adult males and adolescent females are more obvious), using the MMoE model can further exploit this correlation and improve the prediction performance of each task.

[0134] like Figure 8 As shown, the gesture feature encoding generated above can be input as input 801 to the MMoE model, and the MMoE model may include age gating 802 and gender gating 804, as well as a mixed expert network composed of multiple experts (e.g., expert 0 806, expert 1 808, and expert 2 810, but the present invention is not limited thereto), and the outputs are weighted summed and input to the age tower 812 and the gender tower 814, respectively, to generate predictions for age range 815 and gender 817, respectively. For more details about the MoME model, please refer to the paper "Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts" published by Jiaqi Ma et al. in 2018 (e.g., see https: / / dl.acm.org / doi / 10.1145 / 3219819.3220007).

[0135] Alternatively, any other multi-task classification model that can be conceived by those skilled in the art may be adopted.

[0136] When the classification module is a single-task classification module, any applicable single-task classification module conceivable by those skilled in the art may be used.

[0137] The above describes in detail the various components of the machine learning model for predicting the user's age or gender based on the user's gesture information. The following describes an example method and system for predicting the user's age and / or gender using the machine learning model. It should be appreciated that in the following method and system, some implementation details are omitted, and the description of these implementation details can refer to the above description.

[0138] See also Fig. 9 , which shows a flowchart of an example method 900 for predicting the age group or gender of a user according to an embodiment of the present specification. One or more steps of the example method can be, for example, as described above with reference to Figure 3 The described machine learning model 300 is implemented.

[0139] The method 900 may include: at operation 902, a gesture input of a user on a touch panel on a user device may be received.

[0140] For example, when the machine learning model is implemented on the user device, user input from the user's touch panel can be directly received. When the machine learning model is implemented on the server, user input from the user on the application and / or page can be received. Alternatively, the history of gesture input stored on the user device can be received by the server.

[0141] It should be noted that, unlike the posture information (which requires an accelerometer or gyroscope), sound information (which requires a microphone), and motion information (which requires a camera) described above, the above information not only requires additional sensors or dedicated data acquisition equipment, but is also independent of the user's usage process itself, that is, the user may not need to provide the information in order to use the service normally. For example, for online shopping applications, the data from the accelerometer, gyroscope, microphone, and camera are not necessary for the user's online shopping behavior itself. However, in most cases, especially in the currently popular smart devices with touch screens, users themselves need to perform gesture input (i.e., touch input) on the touch screen to use the corresponding services, such as browsing, selecting, and purchasing items in online shopping applications by clicking and sliding.

[0142] Therefore, the embodiments of this specification not only provide another way to predict the user's age group and / or gender, but also, compared with body information, sound information and motion information, the gesture information-based solution of the embodiments of this specification does not rely on additional sensors and / or the collection of additional information.

[0143] In addition, since it does not rely on additional sensors and / or the collection of additional information, less user data needs to be collected, thereby protecting user privacy; the absence of additional sensors also reduces the cost of user equipment; the absence of additional sensors also makes the scope of application of the embodiments of this specification wider, for example, it can be applied to some IoT devices that only have a touch screen; the absence of the need to collect and process additional information also reduces processor occupancy, improves system performance, and reduces power consumption.

[0144] Nevertheless, in a preferred embodiment, the user may be informed that the user's gesture information will be used to predict his or her age group and / or gender. In a more preferred embodiment, the user's explicit authorization may be requested first before the user's gesture information is used.

[0145] Method 900 may also include: at operation 904, gesture information may be generated based on the gesture input of the user, the gesture information including two or more of gesture event type, gesture position, gesture duration, average pressing pressure, average pressing area, gesture timestamp, and associated business identifier. As described above, the gesture event type and gesture position (including the starting point and end point coordinates of the gesture path) may be used as trajectory features of the gesture input, the gesture duration, average pressing pressure, average pressing area, and gesture timestamp may be used as timing features of the gesture input, and the associated business identifier may be used as the category feature of the gesture input.

[0146] The method 900 may further include: at operation 906, using a machine learning model to predict at least one of an age group and a gender of the user based on the gesture information.

[0147] As described above, the machine learning model uses one or more of the trajectory features, timing features, and category features of the gesture input to predict the user's age group. Preferably, the machine learning model uses a combination of the trajectory features, timing features, and category features of the gesture input to predict the user's age group or gender.

[0148] Preferably, the machine learning model may include a trajectory module, and the trajectory module is used to generate a trajectory feature code of the gesture input based on the gesture event type, the gesture starting point coordinates and / or the gesture ending point coordinates. Specifically, the trajectory module can be used to: convert the gesture event type, the gesture starting point coordinates and / or the gesture ending point coordinates into a trajectory image; and generate a trajectory feature code of the gesture input based on the trajectory image using a CNN-based model. For more details of the trajectory module, please refer to the above description of Figure 4 and Figure 5 Description.

[0149] Preferably, the machine learning model may include a timing module, which is used to generate a timing feature encoding of the gesture input based on the gesture duration, average pressing pressure, average pressing area, and / or gesture timestamp. Specifically, the timing module can be used to: generate sampling information using a convolution algorithm based on the gesture duration, average pressing pressure, average pressing area, and / or gesture timestamp; and input the sampling information into an attention mechanism to generate a timing feature encoding of the gesture input. For more details on the timing module, please refer to the above description of the timing module. Figure 6 Description.

[0150] Preferably, the machine learning model may include a category module, the category module being used to: embed the associated business identifier of the gesture input into a word to generate a word embedding representation; generate one or more artificial features based on the gesture information; concatenate the artificial features and the word embedding representation, and input the concatenated output into a wide neural network model to generate the category feature encoding. For more details of the category module, please refer to the above for Figure 7 Description.

[0151] Preferably, the machine learning model may further include a splicing module, which is used to splice the feature codes of two or more of the trajectory features, the time series features and the category features to generate a gesture feature code. When only one feature is used, splicing may not be performed and the machine learning model may not include the splicing module.

[0152] Preferably, the machine learning model may further include: a multi-gated hybrid expert module for predicting both the age group and the gender of the user based on the gesture information. In alternative embodiments, other multi-task classification modules may be used to predict both the age group and the gender. When only one of the age group and the gender is predicted, a single-task classification module may be used to perform the prediction.

[0153] Preferably, before performing the prediction of the age group, the XGBoost model can be used to perform a preliminary division of the user's age group. By performing a preliminary division of the age group, a more accurate age group prediction can be achieved.

[0154] Preferably, at least one of the age group and gender of the user can also be predicted based on the combination of the user's gesture information and one or more of the user's body information, motion information and sound information. It can be understood that the gesture information introduced in the embodiment of this specification can be combined with the body information, motion information and sound information mentioned above, and more types of information can be used to predict the age group or gender, thereby further improving the accuracy of the prediction.

[0155] Preferably, the method 900 may further include: after predicting the age group or gender of the user, determining an identity authentication method for the user or performing identity authentication on the user based on at least one of the predicted age group and gender of the user ( Fig. 9 not shown).

[0156] For example, some young users may not have an ID card yet, or may not have enough knowledge to complete identity authentication. In this case, targeted prompts or guidance can be provided to young users. For example, young users can be prompted that they do not need a physical ID card, and can find their ID card number on the household registration page. For another example, young users can be prompted to request parental assistance or guarantee to perform identity authentication.

[0157] For another example, some elderly users may have poor eyesight. Therefore, when an elderly user is identified, the displayed font and icon can be enlarged, and the authentication face scanning time can be extended.

[0158] Preferably, the method 900 may further include: after predicting the age group or gender of the user, generating a recommendation for the user based on at least one of the predicted age group and gender of the user.

[0159] For example, products or services that are more likely to be of interest to teenagers may be recommended to teenagers; products or services that are more likely to be of interest to female users may be recommended to female users, etc. Providing recommendations based on age group, gender, or a combination of the two may be performed based on any applicable method that can be conceived by those skilled in the art.

[0160] See also Fig.10 , which shows a schematic block diagram of an example system 1000 for predicting the age group or gender of a user according to an embodiment of this specification. The specific description of each module of the system 1000 can refer to the above for Fig. 9 The method 900 and the above-mentioned Figure 2-Figure 8 The introduced machine learning model and the specific details of its various modules.

[0161] like Fig.10 As shown, the system 1000 may include a gesture input receiving module 1002 for receiving a gesture input from a user on a touch panel of a user device. For details of the operation of this module, reference may be made to the description of operation 902 above.

[0162] The system 1000 may also include a gesture information generation module 1004, which is used to generate gesture information based on the user's gesture input, and the gesture information includes two or more of gesture event type, gesture position, gesture duration, average pressing pressure, average pressing area, gesture timestamp, and associated business identifier. The gesture event type and gesture position (including the starting point and end point coordinates of the gesture path) can be used as trajectory features of the gesture input, the gesture duration, average pressing pressure, average pressing area, and gesture timestamp can be used as timing features of the gesture input, and the associated business identifier can be used as a category feature of the gesture input. For specific details of the operation of this module, please refer to the description of operation 904 above.

[0163] The system 1000 may also include a prediction module 1006 for predicting at least one of the age group and gender of the user based on the gesture information using a machine learning model. The specific details of the operation of this module may refer to the description of operation 906 above. As described above, the machine learning model predicts the age group of the user using one or more of the trajectory features, timing features, and category features of the gesture input. Preferably, the machine learning model predicts the age group or gender of the user using a combination of the trajectory features, timing features, and category features of the gesture input.

[0164] Preferably, the machine learning model may include a trajectory module, and the trajectory module is used to generate a trajectory feature code of the gesture input based on the gesture event type, the gesture starting point coordinates and / or the gesture ending point coordinates. Specifically, the trajectory module can be used to: convert the gesture event type, the gesture starting point coordinates and / or the gesture ending point coordinates into a trajectory image; and generate a trajectory feature code of the gesture input based on the trajectory image using a CNN-based model. For more details of the trajectory module, please refer to the above description of Figure 4 and Figure 5 Description.

[0165] Preferably, the machine learning model may include a timing module, which is used to generate a timing feature encoding of the gesture input based on the gesture duration, average pressing pressure, average pressing area, and / or gesture timestamp. Specifically, the timing module can be used to: generate sampling information using a convolution algorithm based on the gesture duration, average pressing pressure, average pressing area, and / or gesture timestamp; and input the sampling information into an attention mechanism to generate a timing feature encoding of the gesture input. For more details on the timing module, please refer to the above description of the timing module. Figure 6 Description.

[0166] Preferably, the machine learning model may include a category module, the category module being used to: embed the associated business identifier of the gesture input into a word to generate a word embedding representation; generate one or more artificial features based on the gesture information; concatenate the artificial features and the word embedding representation, and input the concatenated output into a wide neural network model to generate the category feature encoding. For more details of the category module, please refer to the above for Figure 7 Description.

[0167] Preferably, the machine learning model may further include a splicing module, which is used to splice the feature codes of two or more of the trajectory features, the time series features and the category features to generate a gesture feature code. When only one feature is used, splicing may not be performed and the machine learning model may not include the splicing module.

[0168] Preferably, the machine learning model may further include: a multi-gated hybrid expert module for predicting both the age group and the gender of the user based on the gesture information. In alternative embodiments, other multi-task classification modules may be used to predict both the age group and the gender. When only one of the age group and the gender is predicted, a single-task classification module may be used to perform the prediction.

[0169] Preferably, the system 1000 may also include an age group classification module ( Fig.10 For example, before performing the prediction of the age group, the age group segmentation module can use the XGBoost model to perform a preliminary segmentation of the user's age group. By performing a preliminary segmentation of the age group, a more accurate age group prediction can be achieved.

[0170] Preferably, at least one of the age group and gender of the user may also be predicted based on a combination of the user's gesture information and one or more of the user's body shape information, movement information and sound information.

[0171] Preferably, the system 1000 may also include a user authentication module ( Fig.10 ), for determining an identity authentication method for the user and / or performing authentication on the user based on at least one of the predicted age group and gender of the user. The specific details of the operation of the user authentication module can be referred to above for Fig. 9 Description.

[0172] Preferably, the system 1000 may also include a recommendation module ( Fig.10 ), for generating recommendations for the user based on at least one of the predicted age group and gender of the user. The specific details of the operation of the recommendation module can be referred to above for Fig. 9 Description.

[0173] Fig.11 A schematic block diagram of an apparatus 1100 for implementing a method according to one or more embodiments of the present specification is shown. The apparatus may be used to implement, for example, any of the methods and systems described herein. The apparatus may also be implemented as any computing device or resource allocation device described herein. The apparatus may include a processor 1110 and a memory 1115, the processor being configured to execute any of the methods described above. The memory may store, for example, gesture information, etc. The memory may also be used to store any instructions, variables, intermediate data, etc. that may be used during the execution of the method.

[0174] The device 1100 may include a network connection element 1125, for example, a network connection device that can be connected to other devices through a wired connection or a wireless connection. The wireless connection can be, for example, a WiFi connection, a Bluetooth connection, a 3G / 4G / 5G network connection, etc. For example, the network connection element can be used to connect to a network to obtain data (such as user interaction history, etc.) and other various data. It is also possible to receive input from a user from other devices or transmit data to other devices for display via the network connection element.

[0175] When the device for implementing the machine learning model is implemented on a user device, the device also includes a touch panel for receiving gesture input from the user. The device may also optionally include other peripheral components 1120, such as input devices (keyboard, mouse), output devices (such as display), etc. Preferably, the device may include a touch screen. For example, the user may input the weight of each target through the input device. Corresponding information, such as resource allocation results, may also be output to the user via the output device.

[0176] Each of these modules may communicate with each other directly or indirectly, for example, via one or more buses (eg, bus 1105 ).

[0177] The device may be a physical computing device or a virtual computing device having computing capabilities. Examples of physical computing devices may include, but are not limited to, mainframes, minicomputers, personal computers, etc. Examples of virtual computing devices may include various types of virtual machines (e.g., vmware virtual machines or virtualbox virtual machines, etc.) or virtual containers (e.g., virtual containers such as docker).

[0178] It should be understood that although the apparatus 1100 shown above is a single computing device, it should be appreciated that the apparatus 1100 may actually be a collection of computing devices, such as a computing device cluster, a virtual machine cluster, a computing cloud, a storage cloud, etc.

[0179] Moreover, the present application also discloses a computer-readable storage medium including computer-executable instructions stored thereon, and when the computer-executable instructions are executed by a processor, the processor is caused to perform the methods of the embodiments described herein.

[0180] In addition, the present application also discloses a device, which includes a processor and a memory storing computer-executable instructions. When the computer-executable instructions are executed by the processor, the processor executes the methods of the embodiments described herein.

[0181] In addition, the present application also discloses a system, which includes a device for implementing the methods of the embodiments described herein.

[0182] It can be understood that the method according to one or more embodiments of the present specification can be implemented by software, firmware or a combination thereof.

[0183] It should be understood that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0184] It should be understood that the above description of specific embodiments of this specification is provided. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0185] It should be understood that an element described in singular form herein or shown in the drawings as only one does not limit the number of the element to one. In addition, modules or elements described or shown as separate herein may be combined into a single module or element, and modules or elements described or shown as single herein may be split into multiple modules or elements.

[0186] It should also be understood that the terms and expressions used herein are for description only, and one or more embodiments of this specification should not be limited to these terms and expressions. The use of these terms and expressions does not mean to exclude any equivalent features of the illustrations and descriptions (or parts thereof), and it should be recognized that various modifications that may exist should also be included in the scope of the claims. Other modifications, changes and substitutions may also exist. Accordingly, the claims should be deemed to cover all such equivalents.

[0187] Similarly, it should be pointed out that although the description has been made with reference to the current specific embodiments, ordinary technicians in this technical field should realize that the above embodiments are only used to illustrate one or more embodiments of this specification, and various equivalent changes or substitutions may be made without departing from the spirit of the invention. Therefore, as long as the changes and modifications to the above embodiments are within the scope of the essential spirit of the invention, they will fall within the scope of the claims of this application.

Claims

1. A method for predicting the age group or gender of a user, comprising: receiving a gesture input from a user on a touch panel on a user device; Generate gesture information based on the gesture input of the user, the gesture information including two or more of gesture event type, gesture position, gesture duration, average pressing pressure, average pressing area, gesture timestamp, and associated service identifier; as well as using a machine learning model to predict at least one of the age group and gender of the user based on the gesture information, wherein the machine learning model uses trajectory features, timing features, and category features of the gesture input to predict at least one of the age group and gender of the user, wherein the machine learning model includes a category module, a trajectory module, and a timing module, The category module is used to: embed the associated service identifier of the gesture input into a word to generate a word embedding representation; generate one or more artificial features based on the gesture information; splice the artificial features and the word embedding representation, and input the spliced ​​output into a wide neural network model to generate a category feature code representing the category feature, The trajectory module is used to: convert the gesture event type, gesture starting point coordinates and / or gesture ending point coordinates into a trajectory image; and generate a trajectory feature code of the gesture input based on the trajectory image using a CNN-based model; The timing module is used to: generate sampling information using a convolution algorithm based on the gesture duration, average pressing pressure, average pressing area, and / or gesture timestamp; and input the sampling information into an attention mechanism to generate a timing feature code of the gesture input, wherein the feature codes of the trajectory feature, the timing feature, and the category feature are concatenated to generate a gesture feature code; Wherein using a machine learning model to predict at least one of the age group and gender of the user based on the gesture information comprises: The multi-gated hybrid expert module of the machine learning model is utilized to predict both the age group and gender of the user based on the gesture feature encoding.

2. The method of claim 1, further comprising: Use the XGBoost model to make a preliminary division of users’ age groups.

3. The method of claim 1, further comprising: At least one of the age group and gender of the user is predicted based on a combination of the gesture information of the user and one or more of the body shape information, motion information and sound information of the user.

4. The method of claim 1, further comprising: An identity authentication method for the user is determined based on at least one of the predicted age group and gender of the user.

5. The method of claim 1, further comprising: Recommendations are generated for the user based on at least one of the predicted age group and gender of the user.

6. A system for predicting the age group or gender of a user, comprising: A gesture input receiving module, used to receive a gesture input from a user on a touch panel on a user device; A gesture information generating module, configured to generate gesture information based on the gesture input of the user, wherein the gesture information includes two or more of gesture event type, gesture position, gesture duration, average pressing pressure, average pressing area, gesture timestamp, and associated service identifier; as well as a prediction module, configured to predict at least one of the age group and gender of the user based on the gesture information by using a machine learning model, wherein the machine learning model predicts at least one of the age group and gender of the user by using trajectory features, timing features, and category features of the gesture input, wherein the machine learning model includes a trajectory module, a timing module, and a category module, The category module is used to: embed the associated service identifier of the gesture input into a word to generate a word embedding representation; generate one or more artificial features based on the gesture information; splice the artificial features and the word embedding representation, and input the spliced ​​output into a wide neural network model to generate a category feature code representing the category feature. The trajectory module is used to: convert the gesture event type, gesture starting point coordinates and / or gesture ending point coordinates into a trajectory image; and generate a trajectory feature code of the gesture input based on the trajectory image using a CNN-based model; The timing module is used to: generate sampling information using a convolution algorithm based on the gesture duration, average pressing pressure, average pressing area, and / or gesture timestamp; and input the sampling information into an attention mechanism to generate a timing feature code of the gesture input, wherein the feature codes of the trajectory feature, the timing feature, and the category feature are concatenated to generate a gesture feature code; Wherein using a machine learning model to predict at least one of the age group and gender of the user based on the gesture information comprises: The multi-gated hybrid expert module of the machine learning model is utilized to predict both the age group and gender of the user based on the gesture feature encoding.

7. A device for predicting the age group or gender of a user, comprising: Memory; as well as A processor, wherein the processor is configured to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing instructions, wherein when the instructions are executed by a computer, the computer performs the method according to any one of claims 1 to 5.