Interface interaction method and system based on user equipment holding state, electronic equipment, storage medium and computer program product

Abstract

The invention provides an interface interaction method and system based on a user equipment holding state, electronic equipment, a storage medium and a computer program product. The interface interaction method comprises the following steps: collecting initial inertial sensor data and user gesture sliding curve data of a user equipment end, wherein the initial inertial sensor data at least comprises gravitational acceleration data and angular velocity data; training an initial prediction model according to the initial inertial sensor data and the user gesture sliding curve data to obtain a target prediction model; according to the real-time inertial sensor data of the user equipment side, the user equipment holding state is predicted in real time through the target prediction model, and the user equipment holding state comprises a left hand holding state and a right hand holding state; and determining an arrangement mode of commodity information in the interaction interface of the e-commerce platform according to the holding state of the user equipment, and determining an arrangement mode of interaction buttons in the interaction interface of the e-commerce platform according to the holding state of the user equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of interface design for e-commerce platforms, and in particular to an interface interaction method and system based on the holding state of a user device, an electronic device, a storage medium, and a computer program product. Background Art

[0002] With the rapid development of mobile internet and big data technologies, the user base and transaction volume of e-commerce platforms have grown exponentially. Simultaneously, users' demands for a superior shopping experience on e-commerce platforms are also increasing. This user experience is influenced by a variety of factors, including interface design, interaction habits, and product recommendation algorithms.

[0003] However, the inventors of the present invention found that the current interface interaction methods of e-commerce platforms mostly rely on the same-column list layout and fixed recommendation strategies, and lack optimization of the interface interaction methods of e-commerce platforms from the perspective of bioengineering and human-computer interaction (such as the inability to configure the interaction interface accordingly according to the user's hand habits). As a result, the current interface interaction methods cannot accurately meet the user's shopping needs, thereby affecting the user's shopping experience.

[0004] The contents of the background technology section are merely technologies known to the public and do not necessarily represent the existing technologies in this field. Summary of the Invention

[0005] According to one aspect of the present invention, the present invention provides an interface interaction method based on the holding state of a user device, including: collecting initial inertial sensor data and user gesture sliding curve data from the user device side, the initial inertial sensor data including at least gravity acceleration data and angular velocity data; training an initial prediction model based on the initial inertial sensor data and the user gesture sliding curve data to obtain a target prediction model; predicting the holding state of the user device in real time through the target prediction model based on the real-time inertial sensor data from the user device side, the holding state of the user device including left-hand holding state and right-hand holding state; determining the arrangement of product information in the interactive interface of the e-commerce platform according to the holding state of the user device, and determining the arrangement of interactive buttons in the interactive interface of the e-commerce platform according to the holding state of the user device.

[0006] According to some embodiments of the present invention, training an initial prediction model based on initial inertial sensor data and user gesture sliding curve data to obtain a target prediction model includes: preprocessing the initial inertial sensor data to obtain preprocessed inertial sensor data; determining the holding state of a training user device based on the user gesture sliding curve data; constructing a training data set based on the preprocessed inertial sensor data and the holding state of the training user device; and performing model training on the initial prediction model based on the training data set to obtain a target prediction model.

[0007] According to some embodiments of the present invention, determining the holding state of a training user device based on user gesture sliding curve data includes: determining the coordinate data of the user's initial touch point, the coordinate data of the final touch point, and the coordinate data of the intermediate touch point based on the user gesture sliding curve data; determining first vector data from the coordinate data of the intermediate touch point to the coordinate data of the initial touch point, and determining second vector data from the coordinate data of the intermediate touch point to the coordinate data of the final touch point; and determining the holding state of the training user device based on the first vector data and the second vector data.

[0008] According to some embodiments of the present invention, the arrangement of product information in the interactive interface is determined based on the holding state of the user device, including: when the holding state of the user device is left-handed, high-priority product information is displayed on the right side of the interactive interface; when the holding state of the user device is right-handed, high-priority product information is displayed on the left side of the interactive interface.

[0009] According to some embodiments of the present invention, the arrangement of the interactive buttons in the interactive interface is determined according to the holding state of the user device, including: when the holding state of the user device is a left-hand holding state, the interactive buttons are displayed on the left side of the interactive interface; when the holding state of the user device is a right-hand holding state, the interactive buttons are displayed on the right side of the interactive interface.

[0010] According to another aspect of the present invention, the present invention provides an interface interaction system based on the holding state of a user device, comprising a data acquisition module, a model training module, a state prediction module and an interface configuration module. The data acquisition module collects the initial inertial sensor data and user gesture sliding curve data of the user device end, and the initial inertial sensor data includes at least gravity acceleration data and angular velocity data. The model training module trains an initial prediction model based on the initial inertial sensor data and the user gesture sliding curve data to obtain a target prediction model. The state prediction module predicts the holding state of the user device in real time through the target prediction model based on the real-time inertial sensor data of the user device end, and the holding state of the user device includes the left hand holding state and the right hand holding state. The interface configuration module determines the arrangement of the product information in the interactive interface of the e-commerce platform according to the holding state of the user device, and determines the arrangement of the interactive buttons in the interactive interface of the e-commerce platform according to the holding state of the user device.

[0011] According to some embodiments of the present invention, the model training module preprocesses the initial inertial sensor data to obtain preprocessed inertial sensor data; the model training module determines the holding state of the training user device based on the user gesture sliding curve data; the model training module constructs a training data set based on the preprocessed inertial sensor data and the holding state of the training user device; the model training module performs model training on the initial prediction model based on the training data set to obtain a target prediction model.

[0012] According to some embodiments of the present invention, the model training module determines the coordinate data of the user's initial touch point, the coordinate data of the final touch point, and the coordinate data of the intermediate touch point based on the user gesture sliding curve data; the model training module determines the first vector data from the coordinate data of the intermediate touch point to the coordinate data of the initial touch point, and determines the second vector data from the coordinate data of the intermediate touch point to the coordinate data of the final touch point; the model training module determines the holding state of the training user device based on the first vector data and the second vector data.

[0013] According to some embodiments of the present invention, the interface configuration module displays high-priority product information on the right side of the interactive interface when the user device is held in the left hand; the interface configuration module displays high-priority product information on the left side of the interactive interface when the user device is held in the right hand.

[0014] According to some embodiments of the present invention, the interface configuration module displays the interaction button on the left side of the interaction interface when the user device is held in the left hand; the interface configuration module displays the interaction button on the right side of the interaction interface when the user device is held in the right hand.

[0015] According to yet another aspect of the present invention, an electronic device is provided. The electronic device includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the interface interaction method described above.

[0016] According to another aspect of the present invention, a non-volatile computer-readable storage medium is provided, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, the interface interaction method described above can be implemented.

[0017] According to another aspect of the present invention, a computer program product is provided. The computer program product includes: a computer program stored on a computer-readable storage medium; the computer program includes program instructions, which, when executed by a computer, cause the computer to execute the interface interaction method described above.

[0018] Beneficial effects

[0019] The present invention obtains a target prediction model based on the initial inertial sensor data of the user device and the user gesture sliding curve data. Based on the target prediction model, the user device holding state can be predicted in real time through the real-time inertial sensor data of the user device, and the page layout of the e-commerce platform interaction interface can be determined according to the user device holding state.

[0020] The present invention can optimize the interface interaction method of the e-commerce platform by combining multiple perspectives such as bioengineering and human-computer interaction, and can dynamically adjust the page layout of the e-commerce platform interaction interface according to the user's hand habits, so as to provide users with a page layout that is more in line with user habits, thereby improving the user's shopping experience, and also improving the user's shopping click-through rate and shopping conversion rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram showing a flow chart of an interface interaction method according to an embodiment of the present invention;

[0023] Figure 2 Another schematic diagram showing a flow chart of the interface interaction method according to an embodiment of the present invention;

[0024] Figure 3 Another schematic diagram showing a flow chart of the interface interaction method according to an embodiment of the present invention;

[0025] Figure 4 A structural diagram of an interface interaction system according to an embodiment of the present invention is shown.

[0026] Description of reference numerals:

[0027] Interface interaction system 1; data acquisition module 10; model training module 20; state prediction module 30; interface configuration module 40. DETAILED DESCRIPTION

[0028] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. Like reference numerals in the drawings represent like or similar parts, and thus repeated description thereof will be omitted.

[0030] The described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of these specific details, or other methods, components, materials, devices, etc. may be employed. In these cases, well-known structures, methods, devices, implementations, materials or operations will not be shown or described in detail.

[0031] Furthermore, the terms "include," "comprise," and "have," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0032] The terms "first", "second" and the like in the specification, claims and drawings of the present invention are used to distinguish different objects rather than to describe a specific order.

[0033] The following is a clear and complete description of the technical solutions of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the embodiments described are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of the present invention.

[0034] Fitter's law describes the efficiency and difficulty of human limbs (such as fingers and mice) in moving and hitting targets. For example, the time it takes to complete a target selection task is proportional to the size and distance of the target. Visual bias theory describes how users unconsciously prioritize visual attention on specific areas or elements when observing complex scenes.

[0035] The inventors discovered that in the application scenarios of e-commerce platforms, combining Fitt's law and visual bias theory, it can be concluded that users are more inclined to click on areas that are easier to reach, and that users' visual perception will have a left-right bias. For example, the visual attention of left-handed users will be more concentrated on the right visual scene, while the visual attention of right-handed users will be more concentrated on the left visual scene.

[0036] Based on this, according to one aspect of the present invention, the present invention provides an interface interaction method based on the holding state of the user device. The interface interaction method is applied to the e-commerce platform, and the interface interaction method can adaptively optimize the e-commerce platform interaction interface according to the holding state of the user device.

[0037] Figure 1 A flow chart of the interface interaction method according to an embodiment of the present invention is shown. Figure 1 As shown, the interface interaction method may include steps S100-S400.

[0038] Exemplarily, the interface interaction method may be executed by an interface interaction system having computing capabilities.

[0039] According to an exemplary embodiment, in step S100, the interface interaction system collects initial inertial sensor data and user gesture sliding curve data from the user device. The initial inertial sensor data includes at least gravity acceleration data and angular velocity data.

[0040] For example, a user device includes, but is not limited to, an electronic device such as a mobile phone or tablet computer. The user device can be connected to an e-commerce platform. When a user enters a specific page of the e-commerce platform and browses products by holding the user device, the interface interaction system can collect initial inertial sensor data and user gesture sliding curve data from the user device.

[0041] The initial inertial sensor data may be collected by an inertial sensor disposed on the user device, and the inertial sensor may send the initial inertial sensor data to the interface interaction system.

[0042] For example, the inertial sensor may include a gravity accelerometer and a gyroscope. The gravity accelerometer can collect gravity acceleration data on three orthogonal axes of the user device, and the gravity acceleration data can be used as (A X 、A Y 、A Z). The gyroscope can collect angular velocity data of the user device around three orthogonal axes, which can be expressed as (G X , G Y , G Z )express.

[0043] For example, the sampling frequency of the initial inertial sensor data and the user gesture sliding curve data may be 50-100 Hz, and the interface interaction system may process the data stream in real time using a sliding window method, such as outputting a window containing 10-20 sets of data every 200 ms.

[0044] User gesture sliding curve data can be directly collected by the interface interaction system.

[0045] For example, when a user touches the electronic display screen of the user device and browses products, the interface interaction system obtains the user's gesture sliding curve and determines the corresponding user gesture sliding curve data based on the gesture sliding curve. The user gesture sliding curve data may include data of multiple user touch points on the electronic display screen.

[0046] In step S200, the interface interaction system trains an initial prediction model based on initial inertial sensor data and user gesture sliding curve data to obtain a target prediction model.

[0047] For example, the interface interaction system can perform offline training and verification on the initial prediction model based on the collected initial inertial sensor data and user gesture sliding curve data, and then obtain a target prediction model with prediction function.

[0048] For example, the initial prediction model may be a random forest model. The random forest model is suitable for complex data and can process nonlinear data. By training the random forest model, a random forest model with prediction capability (i.e., a target prediction model) may be obtained.

[0049] According to an example embodiment, after obtaining the target prediction model, the interface interaction system may deploy the target prediction model on the user device.

[0050] For example, the interface interaction system can deploy the trained target prediction model on the user device through the TensorFlow Lite framework (a lightweight machine learning framework optimized for mobile devices and embedded devices) to predict the holding state of the user device.

[0051] Figure 2 Another flowchart of the interface interaction method according to an embodiment of the present invention is shown.

[0052] Alternatively, as Figure 2As shown, step S200 may further include steps S210-S230.

[0053] In step S210 , the interface interaction system preprocesses the initial inertial sensor data to obtain preprocessed inertial sensor data.

[0054] For example, preprocessing includes at least normalization processing and filtering processing.

[0055] Since the dimensions of the initial inertial sensor data are different, the initial inertial sensor data may be normalized. For example, the normalization may be a Min-Max Normalization (linear change) normalization, such as:

[0056]

[0057] Among them, X is the raw data of the initial inertial sensor data, X min is the minimum value of the initial inertial sensor data, X max is the maximum value of the initial inertial sensor data, and X' is the normalized result of the initial inertial sensor data.

[0058] Since the initial inertial sensor data may contain noise, the initial inertial sensor data may be filtered. For example, the filtering process may adopt a sliding average filtering process, such as:

[0059]

[0060] Where t is the serial number representing each set of data in the initial inertial sensor data. Assuming there are 100 sets of data in the initial inertial sensor data, the value of t ranges from 0 to 99.

[0061] After the interface interaction system performs Min-Max normalization processing and sliding average filtering processing on the initial inertial sensor data, preprocessed inertial sensor data can be obtained.

[0062] In step S220, the interface interaction system determines the gripping state of the training user device according to the user gesture sliding curve data.

[0063] For example, the user gesture sliding curve data may include the coordinate data of the user's initial touch point, the coordinate data of the final touch point, and the coordinate data of the intermediate touch points. The interface interaction system can determine the user's device holding state based on the coordinate data of the initial touch point, the coordinate data of the final touch point, and the coordinate data of the intermediate touch points.

[0064] Exemplarily, the user equipment holding state may include a left-hand holding state and a right-hand holding state.

[0065] In step S230, the interface interaction system constructs a training data set based on the pre-processed inertial sensor data and the grip state of the training user device.

[0066] In step S240, the interface interaction system performs model training on the initial prediction model according to the training data set to obtain a target prediction model.

[0067] For example, the interface interaction system uses pre-processed inertial sensor data as input data X and the training user device holding state as target data Y to construct a training data set D.

[0068] For example, the training dataset D can be expressed as:

[0069] D=[(X1, X2, X3, X4, X5, X6), Y];

[0070] As an embodiment, the training data set D may be:

[0071]

[0072] Exemplarily, the interface interaction system may randomly extract 80% of the data in the training data set D as a training set, and take 20% of the data as a validation set to perform model training, thereby obtaining a target prediction model.

[0073] It is understood that because the results of training the user's device grip state obtained through user gesture sliding curve data are not calculated in real time, it is impossible to perceive the user's hand-changing operation in a product browsing scenario in real time. However, the present invention uses the initial inertial sensor data and user gesture sliding curve data to jointly construct a training data set for the training model. The initial inertial sensor data and user gesture sliding curve data can be fused and processed, so that the resulting target prediction model can perform real-time predictions.

[0074] Figure 3 Another flowchart of the interface interaction method according to an embodiment of the present invention is shown.

[0075] Alternatively, as Figure 3 As shown, step S220 may further include steps S221-S223.

[0076] In step S221, the interface interaction system determines the coordinate data of the user's initial touch point, the coordinate data of the final touch point, and the coordinate data of the intermediate touch points based on the user gesture sliding curve data.

[0077] For example, the interface interaction system can record the coordinate data of the initial touch point as (Ax, Ay), the coordinate data of any intermediate touch point as (Bx, By), and the coordinate data of the final touch point as (Cx, Cy) based on the user gesture sliding curve data.

[0078] In step S222, the interface interaction system determines first vector data from the coordinate data of the intermediate touch point to the coordinate data of the initial touch point, and determines second vector data from the coordinate data of the intermediate touch point to the coordinate data of the final touch point.

[0079] For example, the first vector data may be:

[0080] W = (Ax - Bx, Ay - By);

[0081] The second vector data can be:

[0082] V = (Cx-Bx, Cy-By);

[0083] In step S223, the interface interaction system determines the gripping state of the training user device based on the first vector data and the second vector data.

[0084] For example, the interface interaction system calculates the relationship factor Res based on the first vector data and the second vector data. For example:

[0085] Res=(Ay-Cy)[(Ax-Bx)(Cy-By)-(Ay-By)(Cx-Bx)];

[0086] When Res is greater than 0, the interface interaction system determines that the training user device is held in the right hand; otherwise, it is held in the left hand.

[0087] Through the above embodiments, the present invention can accurately judge the user's holding state through the coordinate data of the user's initial touch point on the user device, the coordinate data of the final touch point, and the coordinate data of the intermediate touch point, thereby obtaining the training user device holding state.

[0088] In step S300, the interface interaction system predicts the holding state of the user device in real time through a target prediction model based on the real-time inertial sensor data of the user device.

[0089] For example, the interface interaction system can collect real-time inertial sensor data from the user device in real time. This real-time inertial sensor data may include gravity acceleration data and angular velocity data. The collection method of gravity acceleration data and angular velocity data has been described in detail above and will not be repeated here.

[0090] According to an example embodiment, the interface interaction system can obtain the user device holding state based on the trained target prediction model and taking the real-time inertial sensor data as input.

[0091] In step S400, the interface interaction system determines the arrangement of product information in the e-commerce platform interaction interface according to the holding state of the user device, and determines the arrangement of interaction buttons in the e-commerce platform interaction interface according to the holding state of the user device.

[0092] For example, after determining the holding state of the user device, the interface interaction system configures the layout of the e-commerce platform interaction interface according to the specific holding state corresponding to the holding state of the user device.

[0093] The arrangement of product information may include the display position of the products, etc. The arrangement of interactive buttons may include the display position of the interactive buttons, etc.

[0094] Exemplarily, the interactive buttons include but are not limited to purchase buttons and add-to-cart buttons that can be used to interact with user shopping behaviors.

[0095] Optionally, in step S400, the interface interaction system displays high-priority product information on the right side of the interaction interface when the user device is held in the left hand. Also, the interface interaction system displays high-priority product information on the left side of the interaction interface when the user device is held in the right hand.

[0096] For example, based on the theory of visual bias, when the user device is held in the left hand, the user's visual attention will be more focused on the right visual scene. And when the user device is held in the right hand, the user's visual attention will be more focused on the left visual scene. Therefore, the interface interaction system displays high-priority product information on the right or left side of the interactive interface according to the user device holding state. With such a setting, the present invention can present the main recommended products to the user in the form of key recommendations, thereby improving the user's shopping experience and also increasing the user's shopping click-through rate for the main recommended products.

[0097] Optionally, in step S400, the interface interaction system displays the interaction button on the left side of the interaction interface when the user device is held in the left hand, and displays the interaction button on the right side of the interaction interface when the user device is held in the right hand.

[0098] For example, based on Fitt's law, when the user device is held in the left hand, the user is more inclined to click on the left area of ​​the interactive interface. And when the user device is held in the right hand, the user is more inclined to click on the right area of ​​the interactive interface. Therefore, the interface interaction system displays interactive buttons (such as buy, add to cart, etc.) on the left or right side of the interactive interface according to the holding state of the user device. With such a setting, the present invention can provide users with a page layout that is more in line with user habits, so as to improve the user's shopping experience, and can also improve the shopping conversion rate of user shopping needs.

[0099] Through the above embodiments, the present invention obtains a target prediction model based on the initial inertial sensor data of the user device and the user gesture sliding curve data. Based on the target prediction model, the user device holding state can be predicted in real time through the real-time inertial sensor data of the user device, and the page layout of the e-commerce platform interaction interface can be determined according to the user device holding state.

[0100] The present invention can optimize the interface interaction method of the e-commerce platform by combining multiple perspectives such as bioengineering and human-computer interaction, and can dynamically adjust the page layout of the e-commerce platform interaction interface according to the user's hand habits, so as to provide users with a page layout that is more in line with user habits, thereby improving the user's shopping experience, and also improving the user's shopping click-through rate and shopping conversion rate.

[0101] In addition, some existing technologies for identifying the gripping state of user devices rely on the radio frequency reflection performance of the user device. However, the inventors have found that this identification method can only solve the problem of accidental touches during single-handed operation, and it also suffers from problems such as insufficient recognition accuracy, limited usage scenarios, and susceptibility to interference.

[0102] For example, the RF reflection performance may not provide sufficient differentiation, and it may not be able to accurately distinguish whether it is a left-hand holding state or a right-hand holding state. In particular, confusion may occur when the user alternates hands or holds the device with both hands. For some special holding methods (such as the user's fingers partially covering the antenna, or the palm of the hand lightly touching the frame instead of holding it), the RF reflection performance may not change significantly, resulting in the inability to accurately identify the user device holding state. This recognition method is mainly applicable to static holding states (such as the user's hand is still). For dynamic usage scenarios (such as the user performing gestures on the user device or quickly switching between left and right hands), it may not be able to accurately judge and respond, resulting in misoperation or slow response. In addition, the RF reflection performance is also susceptible to interference from the external environment (such as nearby electromagnetic radiation sources, objects around the human body, other electronic devices, etc.). In addition, the RF reflection performance may be different in different usage environments (such as indoors and outdoors), which can easily lead to the inability to accurately judge the holding state.

[0103] Compared with the prior art technical solution of identifying the holding state of a user device through radio frequency reflection performance, the present invention can perform machine learning training based on initial inertial sensor data and user gesture sliding curve data to obtain a target prediction model. The target prediction model combined with real-time inertial sensor data can accurately predict the holding state of the user device. The present invention will not be interfered with by the external environment and is not prone to misjudgment. The present invention is compatible with multiple usage scenarios. The present invention can also optimize the interface interaction method of the e-commerce platform based on multiple angles such as bioengineering and human-computer interaction, which can improve the user's shopping click-through rate and shopping conversion rate.

[0104] According to another aspect of the present invention, the present invention provides an interface interaction system based on the holding state of a user device, which can adaptively optimize the interactive interface of an e-commerce platform according to the holding state of the user device.

[0105] Figure 4 A structural diagram of an interface interaction system according to an embodiment of the present invention is shown.

[0106] like Figure 4 As shown, the interface interaction system 1 may include a data acquisition module 10, a model training module 20, a state prediction module 30 and an interface configuration module 40.

[0107] According to an exemplary embodiment, the data collection module 10 collects initial inertial sensor data and user gesture sliding curve data from the user device. The initial inertial sensor data includes at least gravity acceleration data and angular velocity data.

[0108] For example, the user device includes but is not limited to electronic devices such as mobile phones or tablet computers. The user device can be connected to an e-commerce platform. When a user enters a specific page of the e-commerce platform by holding the user device and browses products, the data collection module 10 can collect initial inertial sensor data and user gesture sliding curve data of the user device.

[0109] The initial inertial sensor data may be collected by an inertial sensor disposed on the user equipment side, and the inertial sensor may send the initial inertial sensor data to the data collection module 10 .

[0110] For example, the inertial sensor may include a gravity accelerometer and a gyroscope. The gravity accelerometer can collect gravity acceleration data on three orthogonal axes of the user device, and the gravity acceleration data can be used as (A X 、A Y 、A Z ). The gyroscope can collect angular velocity data of the user device around three orthogonal axes, which can be expressed as (G X , GY , G Z )express.

[0111] For example, the sampling frequency of the initial inertial sensor data and the user gesture sliding curve data may be 50-100 Hz. The data acquisition module 10 may process the data stream in real time using a sliding window method, such as outputting a window containing 10-20 sets of data every 200 ms.

[0112] The user gesture sliding curve data can be directly collected by the data collection module 10 .

[0113] For example, when a user touches the electronic display screen of the user device and browses products, the data acquisition module 10 obtains the user's gesture sliding curve and determines the corresponding user gesture sliding curve data based on the gesture sliding curve. The user gesture sliding curve data may include data of multiple user touch points on the electronic display screen.

[0114] The model training module 20 trains an initial prediction model based on the initial inertial sensor data and the user gesture sliding curve data to obtain a target prediction model.

[0115] For example, the model training module 20 can perform offline training and verification on the initial prediction model based on the collected initial inertial sensor data and user gesture sliding curve data, and then obtain a target prediction model with prediction function.

[0116] For example, the initial prediction model may be a random forest model. The random forest model is suitable for complex data and can process nonlinear data. By training the random forest model, a random forest model with prediction capability (i.e., a target prediction model) may be obtained.

[0117] According to an example embodiment, after obtaining the target prediction model, the model training module 20 may deploy the target prediction model on the user device.

[0118] Exemplarily, the model training module 20 can deploy the trained target prediction model on the user device side through the TensorFlow Lite framework (a lightweight machine learning framework optimized for mobile devices and embedded devices) to predict the holding state of the user device.

[0119] Optionally, the model training module 20 preprocesses the initial inertial sensor data to obtain preprocessed inertial sensor data.

[0120] For example, preprocessing includes at least normalization processing and filtering processing.

[0121] Since the dimensions of the initial inertial sensor data are different, the initial inertial sensor data may be normalized. For example, the normalization may be a Min-Max Normalization (linear change) normalization, such as:

[0122]

[0123] Among them, X is the raw data of the initial inertial sensor data, X min is the minimum value of the initial inertial sensor data, X max is the maximum value of the initial inertial sensor data, and X' is the normalized result of the initial inertial sensor data.

[0124] Since the initial inertial sensor data may contain noise, the initial inertial sensor data may be filtered. For example, the filtering process may adopt a sliding average filtering process, such as:

[0125]

[0126] Where t is the serial number representing each set of data in the initial inertial sensor data. Assuming there are 100 sets of data in the initial inertial sensor data, the value of t ranges from 0 to 99.

[0127] After the model training module 20 performs Min-Max normalization processing and sliding average filtering processing on the initial inertial sensor data, pre-processed inertial sensor data can be obtained.

[0128] The model training module 20 determines the holding state of the training user device according to the user gesture sliding curve data.

[0129] For example, the user gesture sliding curve data may include the coordinate data of the user's initial touch point, the coordinate data of the final touch point, and the coordinate data of the intermediate touch points. The model training module 20 can determine the user's device holding state based on the coordinate data of the initial touch point, the coordinate data of the final touch point, and the coordinate data of the intermediate touch points.

[0130] Exemplarily, the user equipment holding state may include a left-hand holding state and a right-hand holding state.

[0131] The model training module 20 constructs a training data set based on the pre-processed inertial sensor data and the grip state of the training user device.

[0132] The model training module 20 performs model training on the initial prediction model according to the training data set to obtain a target prediction model.

[0133] For example, the model training module 20 constructs a training data set D using pre-processed inertial sensor data as input data X and a gripping state of a training user device as target data Y.

[0134] For example, the training dataset D can be expressed as:

[0135] D=[(X1, X2, X3, X4, X5, X6), Y];

[0136] As an embodiment, the training data set D may be:

[0137]

[0138]

[0139] Exemplarily, the model training module 20 may randomly select 80% of the data in the training data set D as a training set, and take 20% of the data as a validation set to perform model training, thereby obtaining a target prediction model.

[0140] It is understood that because the results of training the user's device grip state obtained through user gesture sliding curve data are not calculated in real time, it is impossible to perceive the user's hand-changing operation in a product browsing scenario in real time. However, the present invention uses the initial inertial sensor data and user gesture sliding curve data to jointly construct a training data set for the training model. The initial inertial sensor data and user gesture sliding curve data can be fused and processed, so that the resulting target prediction model can perform real-time predictions.

[0141] Optionally, the model training module 20 determines the coordinate data of the user's initial touch point, the coordinate data of the final touch point, and the coordinate data of the intermediate touch points according to the user gesture sliding curve data.

[0142] For example, the model training module 20 can record the coordinate data of the initial touch point as (Ax, Ay), the coordinate data of any intermediate touch point as (Bx, By), and the coordinate data of the final touch point as (Cx, Cy) based on the user gesture sliding curve data.

[0143] The model training module 20 determines first vector data from the coordinate data of the intermediate touch point to the coordinate data of the initial touch point, and determines second vector data from the coordinate data of the intermediate touch point to the coordinate data of the final touch point.

[0144] For example, the first vector data may be:

[0145] W = (Ax - Bx, Ay - By);

[0146] The second vector data can be:

[0147] V = (Cx-Bx, Cy-By);

[0148] The model training module 20 determines the holding state of the training user device according to the first vector data and the second vector data.

[0149] For example, the model training module 20 calculates the relationship factor Res based on the first vector data and the second vector data. For example:

[0150] Res=(Ay-Cy)[(Ax-Bx)(Cy-By)-(Ay-By)(Cx-Bx)];

[0151] When Res is greater than 0, the model training module 20 determines that the holding state of the training user device is the right-hand holding state, otherwise, it is the left-hand holding state.

[0152] Through the above embodiments, the present invention can accurately judge the user's holding state through the coordinate data of the user's initial touch point on the user device, the coordinate data of the final touch point, and the coordinate data of the intermediate touch point, thereby obtaining the training user device holding state.

[0153] The state prediction module 30 predicts the holding state of the user device in real time through a target prediction model based on the real-time inertial sensor data of the user device.

[0154] For example, the state prediction module 30 may collect real-time inertial sensor data from the user device in real time. The real-time inertial sensor data may include gravity acceleration data and angular velocity data. The collection method of gravity acceleration data and angular velocity data has been described in detail above and will not be repeated here.

[0155] According to an example embodiment, the state prediction module 30 may obtain the holding state of the user device based on the trained target prediction model and taking the real-time inertial sensor data as input.

[0156] The interface configuration module 40 determines the arrangement of product information in the e-commerce platform interactive interface according to the user device holding state, and determines the arrangement of interaction buttons in the e-commerce platform interactive interface according to the user device holding state.

[0157] For example, after determining the holding state of the user device, the interface configuration module 40 configures the layout of the e-commerce platform interaction interface according to the specific holding state corresponding to the holding state of the user device.

[0158] The arrangement of product information may include the display position of the products, etc. The arrangement of interactive buttons may include the display position of the interactive buttons, etc.

[0159] Exemplarily, the interactive buttons include but are not limited to purchase buttons and add-to-cart buttons that can be used to interact with user shopping behaviors.

[0160] Optionally, the interface configuration module 40 displays high-priority product information on the right side of the interactive interface when the user device is held in the left hand. Alternatively, the interface configuration module 40 displays high-priority product information on the left side of the interactive interface when the user device is held in the right hand.

[0161] For example, based on the theory of visual bias, when the user device is held in the left hand, the user's visual attention will be more focused on the right visual scene. And when the user device is held in the right hand, the user's visual attention will be more focused on the left visual scene. Therefore, the interface configuration module 40 displays high-priority product information on the right or left side of the interactive interface according to the user device holding state. With such a setting, the present invention can present the main recommended products to the user in a key recommendation manner, thereby improving the user's shopping experience and also increasing the user's shopping click-through rate for the main recommended products.

[0162] Optionally, the interface configuration module 40 displays the interaction buttons on the left side of the interaction interface when the user device is held in the left hand, and displays the interaction buttons on the right side of the interaction interface when the user device is held in the right hand.

[0163] For example, based on Fitt's law, when the user device is held in the left hand, the user is more inclined to click on the left area of ​​the interactive interface. And when the user device is held in the right hand, the user is more inclined to click on the right area of ​​the interactive interface. Therefore, the interface configuration module 40 displays interactive buttons (such as buy, add to cart, etc.) on the left or right side of the interactive interface according to the user device holding state. With such a setting, the present invention can provide users with a page layout that is more in line with user habits, so as to improve the user's shopping experience, and can also improve the shopping conversion rate of user shopping needs.

[0164] Through the above embodiments, the present invention obtains a target prediction model based on the initial inertial sensor data of the user device and the user gesture sliding curve data. Based on the target prediction model, the user device holding state can be predicted in real time through the real-time inertial sensor data of the user device, and the page layout of the e-commerce platform interaction interface can be determined according to the user device holding state.

[0165] The present invention can optimize the interface interaction method of the e-commerce platform by combining multiple perspectives such as bioengineering and human-computer interaction, and can dynamically adjust the page layout of the e-commerce platform interaction interface according to the user's hand habits, so as to provide users with a page layout that is more in line with user habits, thereby improving the user's shopping experience, and also improving the user's shopping click-through rate and shopping conversion rate.

[0166] According to yet another aspect of the present invention, an electronic device is provided. The electronic device includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the interface interaction method described above.

[0167] According to another aspect of the present invention, a non-volatile computer-readable storage medium is provided, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, the interface interaction method described above can be implemented.

[0168] According to another aspect of the present invention, a computer program product is provided. The computer program product includes: a computer program stored on a computer-readable storage medium; the computer program includes program instructions, which, when executed by a computer, cause the computer to execute the interface interaction method described above.

[0169] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions of the aforementioned embodiments or substitute equivalents for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. An interface interaction method based on the holding state of a user device, characterized in that: The interface interaction method is applied to an e-commerce platform, and the interface interaction method includes: Collecting initial inertial sensor data and user gesture sliding curve data from the user device, wherein the initial inertial sensor data includes at least gravity acceleration data and angular velocity data; Training an initial prediction model based on the initial inertial sensor data and the user gesture sliding curve data to obtain a target prediction model; Predicting a holding state of the user device in real time using the target prediction model according to the real-time inertial sensor data of the user device, where the holding state of the user device includes a left-hand holding state and a right-hand holding state; The arrangement of the product information in the interactive interface of the e-commerce platform is determined according to the holding state of the user device, and the arrangement of the interactive buttons in the interactive interface of the e-commerce platform is determined according to the holding state of the user device.

2. The interface interaction method according to claim 1, characterized in that: The training of the initial prediction model according to the initial inertial sensor data and the user gesture sliding curve data to obtain a target prediction model includes: preprocessing the initial inertial sensor data to obtain preprocessed inertial sensor data; Determining a grip state of a training user device according to the user gesture sliding curve data; Constructing a training data set based on the pre-processed inertial sensor data and the grip state of the training user device; The initial prediction model is trained according to the training data set to obtain the target prediction model.

3. The interface interaction method according to claim 2, characterized in that: The determining the gripping state of the training user device according to the user gesture sliding curve data includes: Determining the coordinate data of the user's initial touch point, the coordinate data of the final touch point, and the coordinate data of the intermediate touch points according to the user gesture sliding curve data; Determine first vector data from the coordinate data of the intermediate touch point to the coordinate data of the initial touch point, and determine second vector data from the coordinate data of the intermediate touch point to the coordinate data of the final touch point; The holding state of the training user device is determined according to the first vector data and the second vector data.

4. The interface interaction method according to claim 1, characterized in that: The method of determining the arrangement of the product information in the interactive interface according to the holding state of the user device includes: When the user device is held in the left hand, high-priority product information is displayed on the right side of the interactive interface; When the user device is held in the right-hand holding state, high-priority product information is displayed on the left side of the interactive interface.

5. The interface interaction method according to claim 1, characterized in that: The determining of an arrangement of interaction buttons in the interaction interface according to the holding state of the user device includes: When the user device is held in the left hand, the interactive button is displayed on the left side of the interactive interface. When the user device is held in the right hand, the interaction button is displayed on the right side of the interaction interface.

6. An interface interaction system based on the holding state of a user device, characterized in that: The interface interaction system includes: A data acquisition module collects initial inertial sensor data and user gesture sliding curve data from the user device, wherein the initial inertial sensor data includes at least gravity acceleration data and angular velocity data; A model training module, which trains an initial prediction model based on the initial inertial sensor data and the user gesture sliding curve data to obtain a target prediction model; A state prediction module, which predicts the holding state of the user device in real time using the target prediction model based on the real-time inertial sensor data of the user device, wherein the holding state of the user device includes a left-hand holding state and a right-hand holding state; An interface configuration module determines the arrangement of product information in the interactive interface of the e-commerce platform according to the holding state of the user device, and determines the arrangement of interactive buttons in the interactive interface of the e-commerce platform according to the holding state of the user device.

7. The interface interaction system according to claim 6, characterized in that: The model training module preprocesses the initial inertial sensor data to obtain preprocessed inertial sensor data; The model training module determines the grip state of the training user device according to the user gesture sliding curve data; The model training module constructs a training data set based on the pre-processed inertial sensor data and the grip state of the training user device; The model training module performs model training on the initial prediction model according to the training data set to obtain the target prediction model.

8. The interface interaction system according to claim 7, characterized in that: The model training module determines the coordinate data of the user's initial touch point, the coordinate data of the final touch point, and the coordinate data of the intermediate touch points according to the user gesture sliding curve data; The model training module determines first vector data from the coordinate data of the intermediate touch point to the coordinate data of the initial touch point, and determines second vector data from the coordinate data of the intermediate touch point to the coordinate data of the final touch point; The model training module determines the holding state of the training user device based on the first vector data and the second vector data.

9. The interface interaction system according to claim 6, characterized in that: The interface configuration module displays high-priority product information on the right side of the interactive interface when the user device is held in the left-hand holding state; The interface configuration module displays high-priority product information on the left side of the interactive interface when the user device is held in the right-hand holding state.

10. The interface interaction system according to claim 6, characterized in that: The interface configuration module displays the interaction button on the left side of the interaction interface when the user device is held in the left hand; The interface configuration module displays the interaction button on the right side of the interaction interface when the user equipment is held in the right hand.

11. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the interface interaction method as described in any one of claims 1-5.

12. A non-volatile computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the interface interaction method according to any one of claims 1 to 5 is implemented.

13. A computer program product, characterized in that The computer program comprises a computer program stored on a computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer is caused to execute the interface interaction method according to any one of claims 1 to 5.

Images