Interface display method and device, equipment, medium and program product

Abstract

The invention provides an interface display method which can be applied to the technical field of computers or the field of finance. The interface display method comprises the following steps: acquiring a first user micro-expression; environment information is obtained, and the environment information comprises real-time weather information; obtaining an interface color matching scheme based on the first user micro-expression, the environment information and a user transaction type; and interface display is carried out based on the interface color matching scheme, and the interface color matching scheme is configured based on hue saturation. The invention further provides an interface display device, equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology or finance, and in particular, to an interface display method, device, equipment, medium and program product. Background Art

[0002] With the improvement of the level of social intelligence, users' personalized needs are growing. In the fields of smart device UI, such as mobile phone desktop backgrounds and smart watch dials, rich and colorful color schemes and configuration schemes have become one of the important ways for the new generation of young people to pursue themselves. In the financial industry, meeting the personalized needs of users and providing them with warm services has become an important development requirement for maintaining user relationships, enhancing user stickiness and thus improving business quality and efficiency. How to allow users to trade in accordance with their rational true intentions in a stable emotional state is also an important concern of banks in the field of transaction security. In addition, the original bank service system interface UI color scheme is directly locked at the system code level, and UI replacement will lead to R&D pressure for large-scale code development. Summary of the invention

[0003] In view of the above problems, the present disclosure provides an interface display method, apparatus, device, medium and program product.

[0004] According to a first aspect of the present disclosure, there is provided an interface display method, characterized in that it includes: obtaining a first user micro-expression; obtaining environmental information, wherein the environmental information includes real-time weather information; obtaining an interface color scheme based on the first user micro-expression, the environmental information and the user transaction type; and displaying an interface based on the interface color scheme, wherein the interface color scheme is configured based on hue and saturation.

[0005] According to an embodiment of the present disclosure, the real-time weather information includes at least one of meteorological information, temperature information and air quality information.

[0006] According to an embodiment of the present disclosure, obtaining an interface color scheme based on the first user micro-expression, the environmental information and the user transaction type includes: obtaining user login information, and determining the user transaction type based on the user login information; classifying and identifying the obtained first user micro-expression; comparing the classification identification information of the first user micro-expression, the environmental information and the user transaction type with pre-stored interface color data table information to obtain the interface color scheme, wherein the classification identification information of the first user micro-expression is associated with the user's emotional state.

[0007] According to an embodiment of the present disclosure, after the interface is displayed based on the interface color scheme, the method also includes: obtaining a second user micro-expression; obtaining classification identification information of the first user micro-expression and classification identification information of the second user micro-expression based on the first user micro-expression and the second user micro-expression, respectively; comparing the classification identification information of the first user micro-expression with the classification identification information of the second user micro-expression; and evaluating the interface color scheme based on the comparison result; wherein, the second user micro-expression is obtained after the user completes the transaction, and the first user micro-expression is obtained before the user starts the transaction.

[0008] According to an embodiment of the present disclosure, the classification identification information of the user's micro-expression includes expression positivity, and the evaluation of the interface color scheme based on the comparison result includes: when the expression positivity of the second user's micro-expression is greater than the expression positivity of the first user's micro-expression, judging that the interface color scheme is valid.

[0009] According to an embodiment of the present disclosure, classifying and identifying user micro-expressions includes: extracting multiple user micro-expressions within a preset time range; classifying and identifying user micro-expressions based on a deep learning model to obtain user emotional state information corresponding to each user micro-expression; judging the current user emotion based on the obtained multiple user emotional state information; and constructing classification identification information of user micro-expressions based on the current user emotion judgment result.

[0010] According to an embodiment of the present disclosure, the deep learning model includes a recurrent neural network or a convolutional neural network.

[0011] According to an embodiment of the present disclosure, the user transaction type is associated with at least one of a transaction risk or a marketing demand.

[0012] According to an embodiment of the present disclosure, when the user transaction type is associated with a risky transaction, the interface color scheme is configured based on a low-saturation hue; and / or, when the user transaction type is associated with a marketing activity, the interface color scheme is configured based on a high-saturation hue.

[0013] According to an embodiment of the present disclosure, when the user transaction type is associated with transaction risk, the interface color scheme is configured based on the Morandi color system; and / or, when the user transaction type is associated with a marketing activity, the interface color scheme is configured based on the rainbow color system.

[0014] A second aspect of the present disclosure provides an interface display device, including: a first acquisition module, configured to acquire a first user micro-expression; a second acquisition module, configured to acquire environmental information, wherein the environmental information includes real-time weather information; a calculation module, configured to acquire an interface color scheme based on the first user micro-expression, the environmental information and the user transaction type; and a display module, configured to display an interface based on the interface color scheme, wherein the interface color scheme is configured based on hue saturation.

[0015] According to an embodiment of the present disclosure, the calculation module includes a first acquisition submodule, a classification submodule and a comparison submodule. The first acquisition submodule is configured to obtain user login information and determine the user transaction type based on the user login information. The classification submodule is configured to classify and identify the acquired first user micro-expression. The comparison submodule is configured to compare the classification identification information of the first user micro-expression, the environmental information and the user transaction type with the pre-stored interface color matching data table information to obtain the interface color matching scheme. The classification identification information of the first user micro-expression is associated with the user's emotional state.

[0016] According to an embodiment of the present disclosure, the interface display device further includes a third acquisition module, a processing module, a comparison module and an evaluation module. The third acquisition module is configured to acquire the second user micro-expression. The processing module is configured to acquire the classification identification information of the first user micro-expression and the classification identification information of the second user micro-expression based on the first user micro-expression and the second user micro-expression, respectively. The comparison module is configured to compare the classification identification information of the first user micro-expression with the classification identification information of the second user micro-expression. The evaluation module is configured to evaluate the interface color scheme based on the comparison result. The second user micro-expression is acquired after the user completes the transaction, and the first user micro-expression is acquired before the user starts the transaction.

[0017] According to an embodiment of the present disclosure, the evaluation module includes a determination submodule and an identification submodule. Among them, the determination submodule is configured to determine whether the expression positivity of the second user's micro-expression is greater than the expression positivity of the first user's micro-expression. The identification submodule is configured to determine that the interface color scheme is valid when the expression positivity of the second user's micro-expression is greater than the expression positivity of the first user's micro-expression. When the expression positivity of the second user's micro-expression is less than or equal to the expression positivity of the first user's micro-expression, the interface color scheme is determined to be invalid.

[0018] According to an embodiment of the present disclosure, the classification submodule includes an extraction unit, an identification unit, a judgment unit and a construction unit. The extraction unit is configured to extract multiple user micro-expressions of the user within a preset time range. The identification unit is configured to classify and identify user micro-expressions based on a deep learning model, and obtain user emotional state information corresponding to each user micro-expression. The judgment unit is configured to judge the current user emotion based on the obtained multiple user emotional state information. The construction unit is configured to construct classification identification information of user micro-expressions based on the current user emotion judgment result.

[0019] The third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned interface display method.

[0020] The fourth aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned interface display method.

[0021] The fifth aspect of the present disclosure also provides a computer program product, including a computer program, which implements the above-mentioned interface display method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0023] Figure 1 The application scenario diagram of the interface display method, apparatus, device, medium and program product according to the embodiments of the present disclosure is schematically shown.

[0024] Figure 2 The flowchart of the interface display method according to the embodiment of the present disclosure is schematically shown.

[0025] Figure 3 The flowchart of the method for obtaining the interface color scheme based on the first user micro-expression, the environmental information and the user transaction type according to an embodiment of the present disclosure is schematically shown.

[0026] Figure 4 The flowchart of the method for classifying and identifying user micro-expressions according to an embodiment of the present disclosure is schematically shown.

[0027] Figure 5 A flowchart of a method for classifying and identifying micro-expressions according to a specific embodiment of the present disclosure is schematically shown.

[0028] Figure 6A flowchart of a method for micro-expression recognition according to an example of the present disclosure is schematically shown.

[0029] Figure 7 The flowchart of the method for evaluating feedback effect according to an embodiment of the present disclosure is schematically shown.

[0030] Figure 8 A flowchart of a method for evaluating the interface color scheme based on a comparison result according to an embodiment of the present disclosure is schematically shown.

[0031] Fig. 9 The structural block diagram of the interface display device according to an embodiment of the present disclosure is schematically shown.

[0032] Fig.10 The structural block diagram of a computing module according to an embodiment of the present disclosure is schematically shown.

[0033] Fig.11 The structural block diagram of the interface display device according to some other embodiments of the present disclosure is schematically shown.

[0034] Fig.12 The structural block diagram of the classification submodule according to an embodiment of the present disclosure is schematically shown.

[0035] Fig.13 The structural block diagram of the evaluation module according to the embodiment of the present disclosure is schematically shown.

[0036] Fig.14 A block diagram of an electronic device suitable for implementing the interface display method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0037] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0038] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0039] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0040] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0041] With the improvement of the level of social intelligence, the personalized needs of users are growing, and various service products are beginning to accelerate the transformation towards customization, especially in the fields of smart device UI, such as mobile phone desktop background, smart watch dial, etc., colorful color matching and configuration schemes have become one of the important ways for the new generation of young people to pursue self and boost their emotions. As for color itself, there are currently many international works and academic papers that show the relationship between color and psychology and emotions, that is, everyone's preference and feelings for color are unique, and personalized color selection has an important impact on our emotions. On the other hand, in the financial industry, meeting the personalized needs of users and providing users with warm services has become an important development requirement for maintaining user relationships, enhancing user stickiness and thus improving business quality and efficiency. How to make users trade in accordance with rational true intentions in a stable emotional state is also an important concern of banks in the field of transaction security. In addition, the color matching of the service system interface UI of existing financial institutions is directly locked at the system code level, and UI replacement will lead to R&D pressure for large-scale code development.

[0042] Based on the above-mentioned problems existing in the prior art, an embodiment of the present disclosure provides an interface display method, which is characterized by comprising: obtaining a first user micro-expression; obtaining environmental information, wherein the environmental information includes real-time weather information; obtaining an interface color scheme based on the first user micro-expression, the environmental information and the user transaction type; and displaying the interface based on the interface color scheme, wherein the interface color scheme is configured based on hue and saturation.

[0043] The embodiments of the present disclosure determine the current emotion of the user in all dimensions by establishing internal and external indicators of environmental information and user micro-expressions, and obtain the interface color scheme in combination with the current emotion of the user and the user's transaction type. The interface color scheme is configured according to hues of different saturations to intervene in user emotions with different colors. The embodiments of the present disclosure can automatically realize personalized settings of the service interface UI color according to the effects of different colors on emotions to influence user emotions. The embodiments of the present disclosure can, on the one hand, improve the temperature of personalized services and improve customer stickiness; on the other hand, it can render the UI color scheme in real time based on the user's emotional state, alleviating the R&D pressure caused by large-scale code development due to UI replacement.

[0044] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0045] It should be noted that the interface display method, apparatus, equipment, medium and program product provided in the embodiments of the present disclosure can be used in the field of computer technology, such as interface display, and can also be used in various fields outside the field of computer technology, such as the financial field, etc. The application field of the interface display method, apparatus, equipment, medium and program product provided by the present disclosure is not limited.

[0046] Figure 1 The application scenario diagram of the interface display method, apparatus, device, medium and program product according to the embodiments of the present disclosure is schematically shown.

[0047] like Figure 1 As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0048] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0049] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0050] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0051] It should be noted that the interface display method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the interface display device provided in the embodiment of the present disclosure can generally be set in the server 105. The interface display method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the interface display device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0052] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0053] The following will be based on Figure 1 The scene described by Figure 2 to Figure 8 The interface display method of the disclosed embodiment is described in detail.

[0054] Figure 2 The flowchart of the interface display method according to the embodiment of the present disclosure is schematically shown.

[0055] like Figure 2 As shown, the interface display method of this embodiment includes operations S210 to S240, and the interface display method can be executed by the server 105.

[0056] In operation S210, a first user micro-expression is acquired.

[0057] In operation S220, environmental information is acquired, wherein the environmental information includes real-time weather information.

[0058] In operation S230, an interface color scheme is acquired based on the first user micro-expression, the environmental information, and the user transaction type.

[0059] In operation S240, an interface display is performed based on the interface color scheme.

[0060] According to an embodiment of the present disclosure, the first user micro-expression can reflect the user's emotional state before the transaction, and the environmental information will affect the user's emotions during the transaction. Therefore, the user's current emotions can be judged in all dimensions by establishing internal and external indicators of environmental information and user micro-expressions. Psychological research shows that color has a strong correlation with psychology and emotions. Everyone's preference and feelings for color are unique, and personalized color selection has an important impact on our emotions. In an embodiment of the present disclosure, while perceiving the user's emotions based on the first user micro-expression and environmental information, the user's transaction type can also be associated with the selection process of the interface color scheme. Since the color types required to boost or appease the psychological state of the user in different transaction types are different, introducing the user's transaction type into the color selection can further accurately and effectively boost the user's emotions when the user conducts different types of transactions. In an embodiment of the present disclosure, the interface color scheme is configured based on hue saturation to render colors of different saturations in different user transaction types to achieve effective intervention in user emotions.

[0061] The embodiments of the present disclosure can, on the one hand, enhance personalized service and improve customer stickiness; on the other hand, can render UI color matching in real time based on the user's emotional state, thereby alleviating the R&D pressure caused by large-scale code development due to UI replacement.

[0062] According to an embodiment of the present disclosure, the real-time weather information includes at least one of meteorological information, temperature information and air quality information. In an embodiment of the present disclosure, the meteorological information may include the meteorological type of the current weather, for example, sunny, cloudy, overcast, light rain, heavy rain, thunderstorm, sandstorm, hail, etc. The temperature information may be divided based on the temperature range. For example, a temperature between 15-26 degrees is a pleasant temperature, a temperature below 15 degrees is a cold temperature, and a temperature above 26 degrees is a hot temperature. The air quality information may include air quality levels, such as excellent, good, lightly polluted, moderately polluted, heavily polluted, etc.

[0063] In one example, to obtain weather information, first call the URL of the weather API interface and obtain the current geographic coordinates. Then construct the request parameters, including latitude and longitude and API key, and use the requests library to send an API request to obtain weather data. Parse the API response and extract fields related to weather conditions (such as conditions, temperature, and air quality). Based on the rules: sunny and cloudy days, with temperatures between 15-26 degrees, and air quality of good and above, are good weather; sunny and cloudy days with temperatures above 26 degrees are hot and good weather; cloudy days, rainy days, and other weather, with temperatures below 15 degrees are bad weather; cloudy days, rainy days, and other weather, with temperatures above 26 degrees are hot and bad weather. Determine whether the weather condition is good weather or bad weather, and finally output according to the classification results.

[0064] Figure 3 The flowchart of the method for obtaining the interface color scheme based on the first user micro-expression, the environmental information and the user transaction type according to an embodiment of the present disclosure is schematically shown.

[0065] like Figure 3 As shown, the method of acquiring the interface color scheme based on the first user micro-expression, the environmental information and the user transaction type in this embodiment includes operations S310 to S330.

[0066] In operation S310, user login information is obtained, and the user transaction type is determined based on the user login information.

[0067] In operation S320, the acquired first user micro-expression is classified and identified.

[0068] In operation S330, the classification identification information of the first user's micro-expression, the environmental information, and the user transaction type are compared with the pre-stored interface color matching data table information to obtain the interface color matching scheme.

[0069] According to an embodiment of the present disclosure, user login information may include user transaction page information, and the user transaction type may be determined. The first user micro-expression may be associated with the user emotion type, and a classification identification of the first user micro-expression may be further established based on the user emotion. Correspondingly, the classification identification information of the first user micro-expression is associated with the user's emotional state. For example, user emotions may include smile, happiness, surprise, calmness, concentration, anger, disgust, anxiety, irritability, etc. In some embodiments, the classification identification information of the user's micro-expression includes the positivity of the expression. The positivity of the expression may be the degree of positivity of the expression. The positivity of the expression may be assigned to facilitate subsequent judgment and comparison. In a specific embodiment, positive micro-expressions may be assigned positive values, and negative micro-expressions may be assigned negative values. For example, smile, happiness, surprise, calmness, and concentration are positive micro-expressions; anger, disgust, anxiety, and irritability are negative micro-expressions. Further, the expression positivity of the above positive micro-expressions can be surprise>happy>smile>calm, and accordingly, the expression positivity of surprise can be set to 4, the expression positivity of happy can be set to 3, the expression positivity of smile can be set to 2, and the expression positivity of calm can be set to 1. It can be understood that under different emotional states, the psychological process of user transactions is quite different. Environmental external information and user micro-expression internal information can comprehensively affect the user's psychology, and further affect the transaction status. The successful implementation of different user transaction types has different requirements for the user's transaction psychology. In order to achieve a safe and stable transaction process, an interface color matching data table can be established in advance to set the classification identification information of the first user's micro-expression, environmental information and user transaction type combination and construct a corresponding color matching scheme for each type of combination. Therefore, after extracting the user's current micro-expression, environmental information and user transaction type in real time, the current color matching scheme can be obtained from the interface color matching data table.

[0070] Figure 4 The flowchart of the method for classifying and identifying user micro-expressions according to an embodiment of the present disclosure is schematically shown.

[0071] like Figure 4 As shown, the method for classifying and identifying user micro-expressions in this embodiment includes operations S410 to S440.

[0072] In operation S410, a plurality of user micro-expressions within a preset time range are extracted.

[0073] In operation S420, user micro-expressions are classified and recognized based on the deep learning model to obtain user emotion state information corresponding to each user micro-expression.

[0074] In operation S430, the current user emotion is determined based on the acquired plurality of user emotion state information.

[0075] In operation S440, classification identification information of user micro-expressions is constructed based on the current user emotion judgment result.

[0076] In the embodiment of the present disclosure, the preset time range can be the time range from the user logging into the program to the start of the transaction. It can be understood that the user's micro-expression changes dynamically, and judging the user's emotional state by the micro-expression at a certain point in time may cause misjudgment. By extracting multiple micro-expressions within the preset time range and comprehensively analyzing them, the true emotion of the current user can be accurately identified. Among them, during the comprehensive analysis, the user's emotional state information corresponding to each micro-expression can be counted, and the user's emotional state information with a high degree of repetition can be used as the judgment basis for the classification identification of the current user's micro-expression. For example, in the process from the user logging into the program to the start of the transaction, there are 5 micro-expressions, four of which correspond to calmness and one corresponds to irritability. Therefore, calmness can be used as the emotional type of the current user, and further, positive micro-expressions can be used as the classification identification information of the user's micro-expressions. Further, in the four-level classification of positive micro-expressions of surprise-happiness-smile-calmness, the user's micro-expression is identified as the fourth-level positive micro-expression.

[0077] According to some embodiments of the present disclosure, the deep learning model includes a recurrent neural network or a convolutional neural network. In order to ensure real-time and high efficiency in continuous micro-expression recognition, the embodiments of the present disclosure choose to use a lightweight face detection and key point detection model, as well as a recurrent neural network (RNN) or a convolutional neural network (CNN) with high performance and fast processing capabilities to classify micro-expressions and achieve real-time micro-expression recognition.

[0078] In some specific embodiments, computer vision and image processing technology can be used to perform motion analysis on continuous facial images and compare the differences in facial features between adjacent frames. By detecting and tracking the motion trajectory of facial features, subtle changes in micro-expressions are captured, and based on this motion analysis, a recurrent neural network (RNN) is combined to analyze and classify facial image sequences, learn and identify the emotions or intentions carried by micro-expressions, and classify micro-expressions into positive and negative.

[0079] Figure 5 A flowchart of a method for classifying and identifying micro-expressions according to a specific embodiment of the present disclosure is schematically shown.

[0080] like Figure 5 As shown, the method for micro-expression recognition in this example includes steps 1 to 5.

[0081] Step 1 Preprocessing: Get continuous facial expression frames from the camera or video stream. Preprocess the acquired images, including face detection and key point detection, to ensure that accurate face areas and facial feature points are extracted.

[0082] Step 2 Feature extraction: Extract facial features in each facial expression frame, such as the key point positions of the eyes, eyebrows, mouth and other areas, and calculate the differences in facial features between adjacent frames.

[0083] Step 3: Micro-expression detection: By comparing the differences in facial features between adjacent frames and calculating the displacement and angle changes of feature points, subtle changes in micro-expressions can be identified.

[0084] Step 4: Micro-expression classification: Use recurrent neural networks (RNN) or other deep learning models to classify and identify micro-expressions. Classify the detected micro-expressions into different emotion categories such as happiness, sadness, anger, etc. In order to ensure real-time and high efficiency in continuous micro-expression recognition, a lightweight face detection and key point detection model, as well as a recurrent neural network (RNN) or convolutional neural network (CNN) with high performance and fast processing capabilities are selected to classify micro-expressions and achieve real-time micro-expression recognition.

[0085] Step 5: Continuous recognition: Continuously recognize the user's micro-expressions, continuously obtain continuous facial expression frames, then compare feature differences, classify micro-expressions, and output the results.

[0086] Figure 6 A flowchart of a method for micro-expression recognition according to an example of the present disclosure is schematically shown.

[0087] like Figure 6 As shown, the micro-expression recognition method of this example includes steps S1 to S5.

[0088] Step S1 uses a real-time face detection algorithm, such as SSD (Single Shot Multibox Detector) or MTCNN (Multi-task Cascaded Convolutional Networks), to perform face detection on continuous video frames and determine the position of the face.

[0089] In step S2, in the detected face area, a fast and accurate key point detection algorithm, such as Dlib or MTCNN, is used to extract the positions of facial feature points.

[0090] Step S3 calculates the difference of facial features between adjacent frames and identifies the changes of micro-expressions. An optical flow algorithm can be used to track the movement of facial feature points, or a simple difference method can be used to calculate feature differences.

[0091] Step S4 uses an efficient recurrent neural network (RNN) model, such as LSTM (Long Short-Term Memory) or GRU (Gated Recurrent Unit), to classify and recognize micro-expressions.

[0092] Step S5 continuously performs micro-expression recognition on the user, continuously obtains continuous video frames, and repeats steps S2 to S4.

[0093] According to an embodiment of the present disclosure, the user transaction type is associated with at least one of a risky transaction or a marketing demand. Among them, a risky transaction may include a transaction with a higher risk level. Marketing activities may include but are not limited to business promotion marketing, customer feedback marketing, etc.

[0094] According to an embodiment of the present disclosure, when the user transaction type is associated with transaction risk, the interface color scheme can be configured based on a low-saturation hue, and when the user transaction type is associated with a marketing activity, the interface color scheme can be configured based on a high-saturation hue.

[0095] In the field of color, hue saturation has a significant impact on emotions. Among them, low-saturation hues convey a gentle and calm emotional tone, while high-saturation hues convey a strong and intense emotional tone. When engaging in risky transactions, emotional stability and calmness are more important for the safe and smooth completion of transactions. Therefore, the interface color scheme can be configured based on low-saturation hues. When it comes to marketing activities, color matching can be based on high-saturation hues to boost user emotions.

[0096] According to some specific embodiments of the present disclosure, when the user transaction type is associated with the transaction risk, the interface color scheme can be configured based on the Morandi color system. When the user transaction type is associated with a marketing activity, the interface color scheme can be configured based on the rainbow color system. Among the low-saturation hues, the Morandi color system has a high degree of recognition. Based on the Morandi color, the user can feel calm and relaxed, which can maximize the emotional stability of the user in dealing with risky transactions and prevent operational risks such as black production and mistakes that may be encountered in financial transactions. In the field of customer marketing and product sales, a general basic hue with a high saturation, that is, the seven-color rainbow color of red, orange, yellow, green, cyan, blue and purple, can be used in combination to boost customer emotions. Among them, red represents passion, strength, and vitality; orange represents vitality, happiness, and creativity; red represents enthusiasm, love, and excitement; green represents calmness, harmony, and nature; cyan represents calmness, balance, and indifference; blue represents calmness, meditation, and quietness; and purple represents mystery, creation, and romance.

[0097] When matching Morandi colors, the base color can be converted into Morandi color by using the color space conversion and adjusting the color parameters. That is, the base color is converted from the current color space to the HSV (hue, saturation, value) color space. In terms of hue (H), the hue of Morandi color is generally in the range of blue-green, about 170-190 degrees. Morandi color is converted by adjusting the hue degree based on the base color value.

[0098] Furthermore, when using the basic color, the UI can be arranged with a gradual transparency from top to bottom based on the transparency adjustment. For example, the transparency can be arranged in a range of 20%-70% from top to bottom to make the user's visual experience more stable.

[0099] In a specific embodiment, the interface color matching data table corresponding to risk transactions may be as shown in Table 1.

[0100] Table 1

[0101]

[0102] In a specific embodiment, the interface color matching data table corresponding to the marketing activity may be as shown in Table 2.

[0103] Table 2

[0104] Micro-expression classification and recognition weather Transaction Type Standard UI base color Positive microexpressions Good weather Customer Marketing purple

[0105] (Continued from Table 2)

[0106]

[0107] After obtaining the interface color scheme, you can automatically complete the interface color scheme change of all transaction pages after login through program tasks.

[0108] According to an embodiment of the present disclosure, after the color scheme is replaced, the user's micro-expressions can continue to be captured, and the changing trend of the user's micro-expressions after the color scheme is applied can be observed through machine learning comparison to optimize the interface color data table.

[0109] Figure 7 The flowchart of the method for evaluating feedback effect according to an embodiment of the present disclosure is schematically shown.

[0110] like Figure 7 As shown, the feedback effect evaluation method of this embodiment includes operations S701 to S704.

[0111] In operation S701, a second user micro-expression is acquired.

[0112] In operation S702, classification identification information of the first user micro-expression and classification identification information of the second user micro-expression are acquired based on the first user micro-expression and the second user micro-expression, respectively.

[0113] In operation S703, the classification identification information of the first user's micro-expression is compared with the classification identification information of the second user's micro-expression.

[0114] In operation S704, the interface color scheme is evaluated based on the comparison result.

[0115] According to an embodiment of the present disclosure, the second user micro-expression is obtained after the user completes the transaction, and the first user micro-expression is obtained before the user starts the transaction. It should be understood that the classification and recognition method of the second user micro-expression can be the same as the classification and recognition method of the first user micro-expression. It can be judged whether the interface color scheme effectively intervenes in the user's transaction behavior by comparing the expression positivity of the second user micro-expression and the first user micro-expression.

[0116] Figure 8 A flowchart of a method for evaluating the interface color scheme based on a comparison result according to an embodiment of the present disclosure is schematically shown.

[0117] like Figure 8 As shown, the method for evaluating the interface color scheme based on the comparison result of this example includes at least operations S801 to S802.

[0118] In operation S801, it is determined whether the expression positivity of the second user's micro-expression is greater than the expression positivity of the first user's micro-expression.

[0119] When the expression positivity of the second user's micro-expression is greater than the expression positivity of the first user's micro-expression, operation S802 is performed.

[0120] In operation S802, it is determined whether the interface color scheme is valid.

[0121] It can be understood that when the expression positivity of the second user's micro-expression is less than or equal to the expression positivity of the first user's micro-expression, operation S803 is performed.

[0122] In operation S803, it is determined that the interface color scheme is invalid.

[0123] According to an embodiment of the present disclosure, whether the interface color scheme is effective can be evaluated based on the change in the positivity of the user's micro-expressions. As mentioned above, the positivity of the expression can be assigned to facilitate subsequent judgment and comparison. In a specific embodiment, positive micro-expressions can be assigned positive values, and negative micro-expressions can be assigned negative values. Furthermore, different sizes can be assigned to different micro-expressions based on the difference in the degree of positivity of the same type of micro-expressions (positive or negative). When the value of the positivity of the expression increases, it means that the user's mood has improved and the intervention is effective.

[0124] Based on the above risk control method, the embodiment of the present disclosure also provides an interface display device. Fig. 9 The device is described in detail.

[0125] Fig. 9 The structural block diagram of the interface display device according to an embodiment of the present disclosure is schematically shown.

[0126] like Fig. 9 As shown, the interface display device 800 of this embodiment includes a first acquisition module 810 , a second acquisition module 820 , a calculation module 830 and a display module 840 .

[0127] The first acquisition module 810 is configured to acquire a first user's micro-expression.

[0128] The second acquisition module 820 is configured to acquire environmental information, wherein the environmental information includes real-time weather information.

[0129] The calculation module 830 is configured to obtain an interface color scheme based on the first user micro-expression, the environmental information and the user transaction type.

[0130] The display module 840 is configured to display the interface based on the interface color scheme, wherein the interface color scheme is configured based on hue and saturation.

[0131] Fig.10 The structural block diagram of a computing module according to an embodiment of the present disclosure is schematically shown.

[0132] like Fig.10 As shown, the calculation module 830 of this embodiment includes a first acquisition submodule 8301 , a classification submodule 8302 and a comparison submodule 8303 .

[0133] Among them, the first acquisition submodule 8301 is configured to obtain user login information and determine the user transaction type based on the user login information.

[0134] The classification submodule 8302 is configured to classify and identify the acquired first user micro-expression.

[0135] The comparison submodule 8303 is configured to compare the classification identification information of the first user's micro-expression, the environmental information and the user transaction type with the pre-stored interface color matching data table information to obtain the interface color matching scheme. The classification identification information of the first user's micro-expression is associated with the user's emotional state.

[0136] Fig.11 The structural block diagram of the interface display device according to some other embodiments of the present disclosure is schematically shown.

[0137] like Fig.11 As shown, the interface display device 800 of this embodiment includes not only a first acquisition module 810, a second acquisition module 820, a calculation module 830 and a display module 840, but also a third acquisition module 850, a processing module 860, a comparison module 870 and an evaluation module 880.

[0138] The third acquisition module 850 is configured to acquire the micro-expression of the second user.

[0139] The processing module 860 is configured to obtain classification identification information of the first user micro-expression and classification identification information of the second user micro-expression based on the first user micro-expression and the second user micro-expression, respectively.

[0140] The comparison module 870 is configured to compare the classification identification information of the first user's micro-expression with the classification identification information of the second user's micro-expression.

[0141] The evaluation module 880 is configured to evaluate the interface color scheme based on the comparison result. The second user micro-expression is obtained after the user completes the transaction, and the first user micro-expression is obtained before the user starts the transaction.

[0142] Fig.12 The structural block diagram of the classification submodule according to an embodiment of the present disclosure is schematically shown.

[0143] like Fig.12 As shown, the classification submodule 8302 of this embodiment includes an extraction unit 83021, an identification unit 83022, a judgment unit 83023 and a construction unit 83024.

[0144] The extraction unit 83021 is configured to extract multiple user micro-expressions of the user within a preset time range.

[0145] The recognition unit 83022 is configured to classify and recognize user micro-expressions based on a deep learning model, and obtain user emotional state information corresponding to each user micro-expression.

[0146] The judgment unit 83023 is configured to judge the current user emotion based on the acquired multiple user emotion state information.

[0147] The construction unit 83024 is configured to construct classification identification information of the user's micro-expression based on the current user emotion judgment result.

[0148] Fig.13 The structural block diagram of the evaluation module according to the embodiment of the present disclosure is schematically shown.

[0149] like Fig.13 As shown, the evaluation module 880 of this embodiment includes a determination submodule 8801 and a recognition submodule 8802 .

[0150] The determination submodule 8801 is configured to determine whether the expression positivity of the second user's micro-expression is greater than the expression positivity of the first user's micro-expression.

[0151] The identification submodule 8802 is configured to determine that the interface color scheme is valid when the expression positivity of the second user's micro-expression is greater than the expression positivity of the first user's micro-expression, and to determine that the interface color scheme is invalid when the expression positivity of the second user's micro-expression is less than or equal to the expression positivity of the first user's micro-expression.

[0152] According to an embodiment of the present disclosure, the first acquisition module 810, the second acquisition module 820, the calculation module 830, the display module 840, the third acquisition module 850, the processing module 860, the comparison module 870, the evaluation module 880, the first acquisition submodule 8301, the classification submodule 8302, the comparison submodule 8303, the determination submodule 8801, the identification submodule, the extraction unit 83021, the identification unit 83022, the judgment unit 83023 and the construction unit 83024 can be combined in one module, or one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first acquisition module 810, the second acquisition module 820, the calculation module 830, the display module 840, the third acquisition module 850, the processing module 860, the comparison module 870, the evaluation module 880, the first acquisition sub-module 8301, the classification sub-module 8302, the comparison sub-module 8303, the determination sub-module 8801, the identification sub-module, the extraction unit 83021, the identification unit 83022, the judgment unit 83023 and the construction unit 83024 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or can be implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the first acquisition module 810, the second acquisition module 820, the calculation module 830, the display module 840, the third acquisition module 850, the processing module 860, the comparison module 870, the evaluation module 880, the first acquisition sub-module 8301, the classification sub-module 8302, the comparison sub-module 8303, the determination sub-module 8801, the identification sub-module, the extraction unit 83021, the identification unit 83022, the judgment unit 83023 and the construction unit 83024 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is executed.

[0153] Fig.14 A block diagram of an electronic device suitable for implementing the interface display method according to an embodiment of the present disclosure is schematically shown.

[0154] like Fig.14As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0155] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0156] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the I / O interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0157] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0158] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus, or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0159] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the item recommendation method provided by the embodiment of the present disclosure.

[0160] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 901. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0161] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0162] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0163] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0164] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0165] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0166] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. An interface display method, characterized in that: include: Get the first user's micro-expression; Acquiring environmental information, wherein the environmental information includes real-time weather information; Acquire an interface color scheme based on the first user micro-expression, the environmental information, and the user transaction type; and Perform interface display based on the interface color scheme, Wherein, the interface color scheme is configured based on hue and saturation.

2. The method according to claim 1, wherein: The real-time weather information includes at least one of meteorological information, temperature information and air quality information.

3. The method according to claim 1, wherein: Acquiring an interface color scheme based on the first user micro-expression, the environmental information, and the user transaction type includes: Obtain user login information, and determine the user transaction type based on the user login information; Classify and identify the acquired micro-expression of the first user; Compare the classification identification information of the first user's micro-expression, the environmental information, and the user transaction type with the pre-stored interface color matching data table information to obtain the interface color matching scheme, Among them, the classification identification information of the first user's micro-expression is associated with the user's emotional state.

4. The method according to claim 1, wherein: After performing interface display based on the interface color scheme, the method further includes: Obtain micro-expressions of the second user; Acquire classification identification information of the first user micro-expression and classification identification information of the second user micro-expression based on the first user micro-expression and the second user micro-expression respectively; Comparing the classification identification information of the first user's micro-expression with the classification identification information of the second user's micro-expression; Evaluate the interface color scheme based on the comparison results; The second user micro-expression is obtained after the user completes the transaction, and the first user micro-expression is obtained before the user starts the transaction.

5. The method according to claim 4, wherein: The classification identification information of the user's micro-expression includes the positivity of the expression, and the evaluation of the interface color scheme based on the comparison result includes: When the expression positivity of the second user's micro-expression is greater than the expression positivity of the first user's micro-expression, it is determined that the interface color scheme is valid.

6. The method according to any one of claims 3 to 5, wherein: Classification and identification of user micro-expressions include: Extract multiple micro-expressions of users within a preset time range; Classify and identify user micro-expressions based on a deep learning model to obtain user emotional state information corresponding to each user micro-expression; Determining the current user emotion based on the acquired multiple user emotion state information; and Construct the classification identification information of user micro-expressions based on the current user emotion judgment results.

7. The method according to claim 6, wherein: The deep learning model includes a recurrent neural network or a convolutional neural network.

8. The method according to claim 1, wherein: The user transaction type is associated with at least one of a transaction risk or a marketing need.

9. The method according to claim 8, wherein: When the user transaction type is associated with risky transactions, the interface color scheme is configured based on a low-saturation hue; and / or when the user transaction type is associated with a marketing activity, the interface color scheme is configured based on a high-saturation hue.

10. The method according to claim 9, wherein: When the user transaction type is associated with transaction risk, the interface color scheme is configured based on the Morandi color system; And / or, when the user transaction type is associated with a marketing campaign, the interface color scheme is configured based on a rainbow color scheme.

11. An interface display device, characterized in that: include: A first acquisition module, configured to acquire a first user's micro-expression; A second acquisition module is configured to acquire environmental information, wherein the environmental information includes real-time weather information; a calculation module configured to obtain an interface color scheme based on the first user micro-expression, the environmental information, and the user transaction type; and A display module is configured to display the interface based on the interface color scheme. Wherein, the interface color scheme is configured based on hue and saturation.

12. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 10.

Images