House recommendation method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of artificial intelligence, and provides a house recommendation method and device, electronic equipment and a storage medium. The method determines a target feature image that a user is interested in by acquiring visual focusing features when the user watches a feature house set information, and acquires an emotional parameter when the user watches the target feature image, determines a target feature image that meets a preset recommendation condition as a recommended feature image, and then determines a recommended house according to the recommended feature image, so that automatic house recommendation is realized, and the recommendation efficiency is improved. Meanwhile, the visual focusing information of the user and the emotion of the user are comprehensively utilized for house recommendation, and the recommendation accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for recommending houses. Background Technology

[0002] Home recommendation is a crucial supporting technology in the real estate industry. It aims to recommend homes that meet users' personalized needs and preferences, thereby improving conversion rates and increasing the success rate and efficiency of users finding satisfactory properties. Accurately capturing user preferences is a key factor in determining the effectiveness of home recommendations. Currently, quantifying user preferences mainly relies on questionnaires, browsing history analysis, and user rating analysis. However, user actions such as filling out forms, browsing, and rating do not accurately reflect spontaneous user preferences and are inefficient, failing to collect large amounts of questionnaires, browsing history, and user ratings in a short period.

[0003] Therefore, there is an urgent need for a housing recommendation system that can efficiently acquire users' personalized and spontaneous preferences to solve the above problems. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method, apparatus, electronic device, and storage medium for recommending houses, in order to solve the problems of low efficiency and low accuracy in house recommendation in the prior art.

[0005] A first aspect of this application provides a house recommendation method, including:

[0006] Display information on characteristic housing sets;

[0007] Track the visual focus characteristics of users when viewing a set of feature houses to identify target feature images that users are interested in;

[0008] Acquire the emotional parameters of users when they view target feature images, and determine target feature images whose emotional parameters meet preset recommendation conditions as recommended feature images;

[0009] Recommended houses are determined based on the recommended feature images.

[0010] In some embodiments of this application, tracking the visual focus features of a user when viewing information about a set of feature houses to determine the target feature image of interest to the user includes:

[0011] Determine the user's visual focus at the target time;

[0012] From the image viewed by the user at the target time, crop the image to a preset size with the visual focus as the center to obtain the target feature image;

[0013] The target time is any moment when a user views the set of featured houses.

[0014] In some embodiments of this application, determining the user's visual focus at a target time includes:

[0015] Obtain the user's eye rotation angle at the target time;

[0016] The ray direction vector in the eye coordinate system is determined based on the eye rotation angle.

[0017] Based on the calibration relationship between the eye coordinate system and the display coordinate system, the ray direction vector is transformed to the display coordinate system to obtain the line-of-sight equation in the display coordinate system;

[0018] The intersection of the line-of-sight equation and the plane of the set of buildings displaying features is determined as the visual focal point.

[0019] In some embodiments of this application, obtaining emotional parameters of a user when viewing a target feature image includes:

[0020] Acquire EEG data when a user views an image featuring a target feature;

[0021] Baseline data is removed from the EEG data to obtain effective EEG data;

[0022] Effective EEG data is input into a pre-trained emotion perception model to obtain emotion parameters.

[0023] In some embodiments of this application, the pre-trained emotion perception model includes a multi-scale temporal feature extraction network, a self-attention feature extraction network, and an emotion estimation network;

[0024] The multi-scale temporal feature extraction network includes multiple convolutional layers and a multi-scale fusion module. The multiple convolutional layers extract features from the effective EEG data respectively, and the multi-scale fusion module fuses the features extracted by each convolutional layer to obtain multi-scale features.

[0025] The self-attention feature extraction network generates an attention feature matrix based on multi-scale features. The attention feature matrix is ​​used to characterize the relationship between different time steps and different electrodes that acquire EEG data.

[0026] The emotion estimation network maps the attention feature matrix into a two-dimensional vector, processes the two-dimensional vector using a growth curve function to obtain the preference arousal value, and processes the two-dimensional vector using a hyperbolic tangent function to obtain the preference effectiveness value. The emotion parameters include the preference arousal value and the preference effectiveness value.

[0027] In some embodiments of this application, the self-attention feature extraction network generates an attention feature matrix based on multi-scale features, including:

[0028] Determine the feature dimensions and number of sequences for multi-scale features. The number of sequences is the product of the number of electrodes used to collect EEG data and the number of time steps.

[0029] The multi-scale features are embedded and encoded based on the feature dimension and the number of sequences to obtain the input feature matrix;

[0030] Add positional encoding to the input feature matrix;

[0031] The multi-head self-attention mechanism is used to extract features from the input feature matrix with added positional encoding to obtain the attention feature matrix.

[0032] In some embodiments of this application, determining that the emotion parameter meets preset recommended conditions includes:

[0033] Obtain the preference arousal value and preference effectiveness value from the emotion parameters;

[0034] If the preference arousal value is greater than the first preset threshold and the preference efficacy value is greater than the second preset threshold, then the emotion parameters are determined to meet the preset recommendation conditions.

[0035] A second aspect of this application provides a house recommendation device, comprising:

[0036] The display module is configured to display information about a set of characteristic houses;

[0037] The visual feature extraction module is configured to track the visual focus features of a user when viewing a set of feature houses, and to determine the target feature image that the user is interested in.

[0038] The emotion feature extraction module is configured to acquire the emotion parameters of a user when viewing a target feature image, and determine the target feature image whose emotion parameters meet the preset recommendation conditions as the recommended feature image.

[0039] The recommendation module is configured to determine recommended houses based on the recommendation feature image.

[0040] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0041] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0042] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment obtains the visual focus features when a user views the feature house set information, determines the target feature image that the user is interested in, and obtains the emotional parameters when the user views the target feature image. The target feature image whose emotional parameters meet the preset recommendation conditions is determined as the recommended feature image. Then, the recommended house is determined based on the recommended feature image, which can realize automatic house recommendation and improve recommendation efficiency. At the same time, by comprehensively utilizing the user's visual focus information and user emotions for house recommendation, the recommendation accuracy is improved. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic flowchart of a house recommendation method provided in an embodiment of this application.

[0045] Figure 2 This is a flowchart illustrating a method for determining target feature images of interest to a user, as provided in an embodiment of this application.

[0046] Figure 3 This is a flowchart illustrating the method for determining a user's visual focus at a target time, as provided in an embodiment of this application.

[0047] Figure 4 This is a flowchart illustrating the method for obtaining emotional parameters when a user views a target feature image, as provided in an embodiment of this application.

[0048] Figure 5 This is a flowchart illustrating the method for generating an attention feature matrix based on multi-scale features using a self-attention feature extraction network provided in this application embodiment.

[0049] Figure 6 This is a flowchart illustrating the method for determining whether emotion parameters meet preset recommendation conditions, as provided in an embodiment of this application.

[0050] Figure 7 This is a block diagram of a system for implementing the house recommendation method provided in the embodiments of this application.

[0051] Figure 8 This is a structural block diagram of the emotion perception model provided in the embodiments of this application.

[0052] Figure 9This is a schematic diagram of an image of interest obtained from a head-mounted display screen based on a user visual focus tracking system provided in an embodiment of this application.

[0053] Figure 10 This is a schematic diagram of a house recommendation device provided in an embodiment of this application.

[0054] Figure 11 This is a schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0055] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0056] A method and apparatus for recommending houses according to embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0057] As mentioned above, currently, in order to quantify user preferences, the main methods used are questionnaires, browsing history analysis, and user rating analysis. However, user behaviors such as filling out forms, browsing, and rating cannot accurately reflect users' spontaneous preferences, and are inefficient, making it impossible to collect a large number of questionnaires, browsing history, and user ratings in a short period of time.

[0058] In view of this, this application provides a house recommendation method. By acquiring the visual focus features of a user when viewing a set of feature houses, the method determines the target feature image that the user is interested in, and acquires the emotional parameters of the user when viewing the target feature image. The method determines the target feature image whose emotional parameters meet the preset recommendation conditions as the recommended feature image, and then determines the recommended house based on the recommended feature image. This method can realize automatic house recommendation and improve recommendation efficiency. At the same time, by comprehensively utilizing the user's visual focus information and user emotions for house recommendation, the method improves recommendation accuracy.

[0059] Figure 1 This is a schematic flowchart of a house recommendation method provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0060] In step S101, the feature house set information is displayed.

[0061] In step S102, the visual focus features of the user when viewing the set of feature houses are tracked to determine the target feature image that the user is interested in.

[0062] In step S103, the emotional parameters of the user when viewing the target feature image are obtained, and the target feature image whose emotional parameters meet the preset recommendation conditions is determined as the recommended feature image.

[0063] In step S104, recommended houses are determined based on the recommended feature image.

[0064] In some embodiments of this application, the method can be executed by a terminal device with certain processing capabilities. This terminal device may include or be connected to a display device used to display feature house set information to the user.

[0065] In one example, the feature house set information can be a pre-set house information set, such as house images or videos including different types of houses. A general house information set can be set, or different house information sets can be set for different users according to user requirements.

[0066] In some embodiments of this application, the terminal device may further include or be connected to a user eye-tracking module. This eye-tracking module is used to track the user's eye movements while viewing the feature house set information, obtain the user's visual focus features when viewing the feature house set, and then determine the target feature image of interest to the user. The target feature image is the house image at the user's visual focus point when viewing the feature house set information.

[0067] In some embodiments of this application, the terminal device may further include or be connected to a brain-computer interface (BCI) capable of collecting user electroencephalogram (EEG) data. The terminal device can extract EEG data from the user while viewing a target feature image and determine the user's emotional parameters based on this EEG data. These emotional parameters characterize the user's level of interest in the target feature image.

[0068] If the emotion parameter meets the preset recommendation conditions, that is, the user's interest in the target feature image is greater than the preset threshold, then the target feature image can be determined as a recommended feature image, and the target feature image can be added to a set of recommended feature images.

[0069] The system can iteratively execute the following operations while the user is viewing the feature house set information: obtaining the target feature image, determining the user's emotional parameters when viewing the target feature image, and adding the target feature image to the recommended feature image set when the emotional parameters meet the preset recommendation conditions, until the user has viewed all the content in the feature house set information, or the number of images in the recommended feature image set reaches the preset number threshold.

[0070] Terminal devices can determine recommended houses based on recommended feature images. In one example, similar features can be extracted from each image in the recommended feature image set, and the house to be recommended to the user can be determined based on these similar features.

[0071] For example, a multimodal housing database can be configured in the terminal device, containing all images, text descriptions, and quantitative data of the properties. Alternatively, the multimodal housing database can be configured to connect to a cloud server, allowing the terminal device to communicate with the cloud server and retrieve data from the multimodal housing database.

[0072] For any given property, all its images can be input into a visual Transformer model to obtain the visual feature set {f} of each salient element in the images. nm}. This can be achieved through the visual feature set {f nm The recommended houses are determined by retrieving similar features from each image in the extracted recommended feature image set {f}. Alternatively, the similarities between the extracted features of each image in the recommended feature image set and the visual feature set {f} can be compared. nm The distances between various features are used to determine recommended houses. In one example, the visual Transformer model can employ a pre-trained DinoV2 model.

[0073] According to the technical solution provided in the embodiments of this application, by acquiring the visual focus features of a user when viewing a set of characteristic houses, the target feature image that the user is interested in is determined, and the emotional parameters of the user when viewing the target feature image are acquired. The target feature image whose emotional parameters meet the preset recommendation conditions is determined as the recommended feature image. Then, the recommended house is determined based on the recommended feature image, which can realize automatic house recommendation and improve recommendation efficiency. At the same time, by comprehensively utilizing the user's visual focus information and user emotions for house recommendation, the recommendation accuracy is improved.

[0074] Figure 2 This is a flowchart illustrating a method for determining target feature images of interest to a user, as provided in an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0075] In step S201, the user's visual focus at the target time is determined.

[0076] In step S202, the target feature image is obtained by cropping a preset size image from the image viewed by the user at the target time, with the visual focus as the center.

[0077] The target time is any moment when a user views the set of featured houses.

[0078] In some embodiments of this application, tracking the visual focus features of a user when viewing the feature house set information and determining the target feature image that the user is interested in can be achieved by first determining the user's visual focus at the target time, and then cropping an image of a preset size from the image viewed by the user at the target time with the visual focus as the center to obtain the target feature image.

[0079] Figure 3 This is a flowchart illustrating a method for determining a user's visual focus at a target time, as provided in an embodiment of this application. Figure 3 As shown, the method includes the following steps:

[0080] In step S301, the user's eye rotation angle at the target time is obtained.

[0081] In step S302, the ray direction vector in the eye coordinate system is determined based on the eye rotation angle.

[0082] In step S303, the ray direction vector is transformed to the display coordinate system according to the calibration relationship between the eye coordinate system and the display coordinate system to obtain the line-of-sight equation of the display coordinate system.

[0083] In step S304, the intersection of the line-of-sight equation and the plane of the set of houses displaying features is determined as the visual focus.

[0084] In some embodiments of this application, when determining the user's visual focus at a target time, the user's eye rotation angle at the target time can be obtained first. In one example, this eye rotation angle can be, for example, a two-dimensional rotation angle (θ). xt θ yt ).

[0085] The ray direction vector in the eye coordinate system can be calculated based on the eye rotation angle. If the eye coordinate system is {E}, the ray direction vector from the origin of the coordinate system {E} to the point (θ) can be determined. xt θ yt The vector of ) is the ray direction vector v. t The origin of the coordinate system {E} can also be denoted as the virtual center position P of the eyeball. EC，t .

[0086] The ray direction vector v can be t and the virtual center position of the eyeball P EC，t Transform to the display coordinate system {D}. The transformation relationship from the eye coordinate system {E} to the display coordinate system {D} is obtained through calibration. Then, the direction vector v... t After transforming to the display coordinate system {D}, the line-of-sight equation in the display coordinate system {D} can be obtained, which is the ray direction in the display coordinate system {D}. Finally, the intersection of this line-of-sight equation and the plane of the set of display feature houses is determined as the visual focus.

[0087] In some implementations, the intersection point P of the line-of-sight equation and the plane displaying the feature house set can also be considered. E，t After distortion correction, the user's visual focus P in the real-time image is obtained. F，t , the P F，t As the visual focus, it refers to the local area that the user's eyes are currently focused on.

[0088] To improve the stability of the user's visual focus, a Kalman filter can also be used for P. F，t Perform filtering.

[0089] When cropping an image to a preset size with the visual focus as the center to obtain the target feature image, P can be used. F，t Centered on the real-time image, a preset size, such as 256×256 pixels, is cropped from the image of interest. I，t .

[0090] Figure 4 This is a flowchart illustrating a method for obtaining emotional parameters of a user when viewing a target feature image, as provided in an embodiment of this application. Figure 4 As shown, the method includes the following steps:

[0091] In step S401, EEG data is acquired when the user views the target feature image.

[0092] In step S402, baseline data is removed from the EEG data to obtain valid EEG data.

[0093] In step S403, valid EEG data is input into a pre-trained emotion perception model to obtain emotion parameters.

[0094] In some embodiments of this application, obtaining the emotional parameters of a user when viewing a target feature image can be achieved by first acquiring the user's electroencephalogram (EEG) data while viewing the target feature image, and then removing the baseline data from the EEG data to obtain the effective EEG data. This is because the baseline data in the EEG data represents the user's EEG signal in a resting state, which has significant individual differences. Therefore, reliable emotion perception can be achieved based on the difference between the real-time acquired EEG data and the resting EEG signal.

[0095] Next, valid EEG data can be input into a pre-trained emotion perception model to obtain emotion parameters.

[0096] In some implementations, the pre-trained emotion perception model may include a multi-scale temporal feature extraction network, a self-attention feature extraction network, and an emotion estimation network.

[0097] The multi-scale temporal feature extraction network includes multiple convolutional layers and a multi-scale fusion module. The multiple convolutional layers extract features from the effective EEG data, and the multi-scale fusion module fuses the features extracted by each convolutional layer to obtain multi-scale features.

[0098] In one example, EEG data x t and baseline data x b The difference x t -x b As input to the emotion perception model, x t -x b The dimensions can be, for example, 32×128×1.

[0099] The multi-scale temporal feature extraction network can include four convolutional layers and a multi-scale fusion module. The kernel size of the first convolutional layer can be 1×7, the number of kernels is 64, the time span is 2, and each convolutional layer is followed by a batch normalization layer and a ReLU layer, outputting the first layer feature h1 with a dimension of 32×64×64.

[0100] The kernel size of the second convolutional layer can be 1×5, the number of kernels is 64, the time span is 2, each convolutional layer is followed by a batch normalization layer and a ReLU layer, and the output dimension is 32×32×64 for the second layer feature h2.

[0101] The kernel size of the third convolutional layer can be 1×3, the number of kernels is 128, the span is 2, each convolutional layer is followed by a batch normalization layer and a ReLU layer, and the output dimension is 32×16×128 for the third layer feature h3.

[0102] The kernel size of the fourth convolutional layer can be 1×3, the number of kernels is 128, the span is 2, each convolutional layer is followed by a batch normalization layer and a ReLU layer, and the output dimension is 32×8×128 for the fourth layer feature h4.

[0103] The multi-scale fusion module first performs average pooling on the fourth-layer features to obtain a global feature h0 with dimensions of 32×1×128. Then, the second-dimensional features h0, h1, h2, h3, and h4 are subjected to bilinear interpolation, all aligned to a size of 32×64, and channel concatenation is performed to obtain a multi-scale feature h with dimensions of 32×64×512. MS Next, two convolutional layers with a kernel size of 1×1, a kernel count of 64, and a span of 1 are used for multi-scale feature fusion. Each convolutional layer is followed by a batch normalization layer and a ReLU layer. Finally, the multi-scale fusion module outputs a 32×64×64 dimensional multi-scale feature h. MF .

[0104] The self-attention feature extraction network generates an attention feature matrix based on multi-scale features. The attention feature matrix is ​​used to characterize the relationship between different time steps and different electrodes that acquire EEG data.

[0105] Figure 5 This is a flowchart illustrating the method for generating an attention feature matrix based on multi-scale features using a self-attention feature extraction network provided in this application embodiment. Figure 5 As shown, the method includes the following steps:

[0106] In step S501, the feature dimension and number of sequences of the multi-scale features are determined.

[0107] The sequence number is the product of the number of electrodes used to collect EEG data and the number of time steps.

[0108] In step S502, the multi-scale features are embedded and encoded based on the feature dimension and the number of sequences to obtain the input feature matrix.

[0109] In step S503, positional encoding is added to the input feature matrix.

[0110] In step S504, a multi-head self-attention mechanism is used to extract features from the input feature matrix with added position encoding to obtain the attention feature matrix.

[0111] In some embodiments of this application, when the self-attention feature extraction network generates an attention feature matrix based on multi-scale features, it can first determine the feature dimension and sequence number of the multi-scale features, and then embed and encode the multi-scale features based on the feature dimension and sequence number to obtain the input feature matrix. Here, the sequence number is the product of the number of electrodes and the number of time steps used to collect EEG data.

[0112] In one example, if the feature dimension is 64, the number of electrodes collecting EEG data is 32, and the time steps are 64, then the number of sequences is 2048. Therefore, the above multi-scale feature h can be analyzed. MF The data is encoded to obtain a 2048×64 dimension format to adapt to the Transformer.

[0113] Next, positional encoding can be added to the input feature matrix. For example, a linear transformation can be used to map the input to the Transformer's embedding space, mapping the 64-dimensional features to 32 dimensions, resulting in a 2048×32-dimensional input matrix. Then, positional encoding is added to generate a 2048×32-dimensional positional encoding matrix, which is then superimposed on the input matrix.

[0114] Finally, a multi-head self-attention mechanism is used to extract features from the input feature matrix with added positional encoding to obtain the attention feature matrix.

[0115] In one example, the input matrix can be divided into multiple heads, each calculating self-attention. First, an attention score is calculated using queries, keys, and values. Then, the values ​​are weighted and summed based on their attention scores to obtain the output of each head. Next, concatenation and linear transformation are performed, concatenating the outputs of all heads and applying a linear transformation to obtain the final output. Finally, a feedforward neural network further processes the output, transforming the self-attention mechanism's output into a 64×64 dimension output matrix h through two fully connected layers. out .

[0116] In some embodiments of this application, the emotion estimation network of the pre-trained emotion perception model maps the attention feature matrix into a two-dimensional vector, processes the two-dimensional vector using a growth curve function to obtain the preference arousal value, and processes the two-dimensional vector using a hyperbolic tangent function to obtain the preference effectiveness value. The emotion parameters include the preference arousal value and the preference effectiveness value.

[0117] The emotion estimation network can consist of two fully connected layers, used to process the output matrix h of the self-attention feature extraction network. out The mapping is a two-dimensional vector. The first dimension of this two-dimensional vector, after passing through the Sigmoid (growth curve) function, yields the preference arousal value A, which can range from [0, 1] and represents the degree of arousal of the preference emotion. The second dimension, after passing through the tanh (hyperbolic tangent) function, yields the preference efficacy value V, which can range from [-1, 1] and represents the preference emotion of liking or disliking.

[0118] In some embodiments of this application, the emotion perception model may include two phases during training: pre-training and fine-tuning. During the pre-training phase, a large amount of general training data T may be collected first. G The aim is to enable the model to learn emotion-related EEG signal features from a large amount of general training data.

[0119] In one example, it is possible to collect EEG signal samples x from each user while multiple users are watching specific video content. train，i Each user's emotional arousal value and preference effectiveness value are scored to obtain training labels y. Atrain，i and y Vtrain，i Using the aforementioned EEG signal samples and training labels, the user emotion perception model is trained in a supervised manner to minimize the error between the model's output label and the training label, thus obtaining a pre-trained general model.

[0120] During the fine-tuning phase, individual training data T can be collected. F The aim is to enable the model to further learn EEG characteristics and emotion estimates relevant to individual users.

[0121] In one example, it is possible to collect a specific user's electroencephalogram (EEG) signal sample x while the user is watching specific video material. ft,i The system scores the emotional arousal value and preference effectiveness of a specific user, and obtains the training label y. Aft,i and y Vft,i Using the aforementioned EEG signal samples and training labels, the general model is fine-tuned to minimize the error between the model's output label and the training label, thus obtaining a personalized emotion perception model for the specific user.

[0122] Figure 6 This is a flowchart illustrating the method for determining whether emotion parameters meet preset recommendation conditions, as provided in an embodiment of this application. Figure 6 As shown, the method includes the following steps:

[0123] In step S601, the preference arousal value and preference effectiveness value in the emotion parameters are obtained.

[0124] In step S602, it is determined that the preference arousal value is greater than the first preset threshold and the preference efficacy value is greater than the second preset threshold, and the emotion parameters are determined to meet the preset recommendation conditions.

[0125] In some embodiments of this application, when determining whether an emotion parameter meets preset recommendation conditions, the preferred arousal value and preferred efficacy value in the emotion parameter can be obtained first. Then, it is determined whether the preferred arousal value is greater than a first preset threshold and whether the preferred efficacy value is greater than a second preset threshold. If it is determined that the preferred arousal value is greater than the first preset threshold and the preferred efficacy value is greater than the second preset threshold, then it is determined that the emotion parameter meets the preset recommendation conditions.

[0126] The first preset threshold can be set to 0.5, for example, and the second preset threshold can be set to 0.75, for example.

[0127] Figure 7 This is a block diagram of a system for implementing the house recommendation method provided in the embodiments of this application. For example... Figure 7 As shown, the system may include a head-mounted display, which comprises a brain-computer interface module, an eye-tracking module, and a display module. Based on the head-mounted display, users can immerse themselves in watching video content, and their electroencephalogram (EEG) and eye-tracking physiological signals can be collected synchronously.

[0128] The brain-computer interface module can have 32 surface-mount electrodes, enabling simultaneous acquisition of brain waves at a sampling rate of 1024 Hz. A 1-75 Hz bandpass filter is used to filter the raw EEG signal, which is then downsampled to 128 Hz. EEG data x is obtained within a 1-second time window. t The dimensions are 32×128.

[0129] The eye-tracking module consists of an infrared light source and a miniature camera, which can capture pupil images and obtain the two-dimensional rotation angle (θ) of the eyeball by measuring the angle of the pupil. x θ y ).

[0130] The display module consists of a microOLED and a lens module, which can project video images onto the user's left and right eyes respectively to form a three-dimensional stereoscopic display. The image at time t is I. t .

[0131] The brain-computer interface module outputs EEG data to the user's emotion perception model, which calculates the user's preferred arousal level and preference effectiveness based on the EEG data. The eye-tracking module outputs eye orientation to the user's visual focus tracking system, which outputs the image of interest from the video frame displayed on the display module.

[0132] Based on preference arousal and preference effectiveness, it can be determined whether the image of interest is a recommended positive sample. If so, it is saved to the recommended positive sample library, and the top K positive samples from the library are used to train the visual feature set in the visual Transformer. Here, K is a positive integer.

[0133] By comparing the retrieved visual feature set with various data in the multimodal housing database using distance measurement and condition filtering, recommended houses can be obtained.

[0134] In other words, when recommending houses, users can wear the aforementioned head-mounted display, and the display module can play videos containing house content for the user. During the user's viewing, the system acquires the image of interest (I) output by the user's visual focus tracking system. I,t And the current preference arousal level A output by the user emotion perception model. t With preference effectiveness V t .

[0135] When the user's preference arousal level A t The value is greater than the first preset threshold of 0.5, and the preference effectiveness value V t If the value is greater than the second preset threshold of 0.75, then the current image of interest is considered to be I. I,t It contains house elements that evoke intrinsically positive emotions in users. At this point, the currently interesting image can be stored in the positive recommendation sample library P.

[0136] When the number of images in the recommended positive sample library exceeds a third preset threshold (e.g., 10), the visual Transformer model can be used to extract features from the 10 images with the highest preference effectiveness in the positive sample library, and obtain the retrieval visual feature set {f}. nm'}.

[0137] By retrieving the visual feature set {f nm The system performs a cosine distance measurement with the visual feature set in the multimodal housing database, and retrieves recommended housing listings for the user based on the cosine distance. Finally, based on other filtering information provided by the user, such as budget and population, it filters out effective recommended housing listings from the recommended listings and pushes them to the user.

[0138] Figure 8 This is a structural block diagram of the emotion perception model provided in the embodiments of this application. Figure 8 As shown, the emotion perception module includes four convolutional layers, a multi-scale fusion module, a multi-head self-attention module, an emotion estimation network, a sigmoid function, and a tanh function.

[0139] The four convolutional layers are designated as Convolution 1, Convolution 2, Convolution 3, and Convolution 4. The multi-scale fusion module performs multi-scale fusion of the features output from the four convolutional layers. The multi-head self-attention module, comprising six heads, further extracts features from the output of the multi-scale fusion module. The sentiment estimation network estimates the output features of the multi-head self-attention module, outputting preference arousal level via a sigmoid function and preference effectiveness value via a tanh function.

[0140] Figure 9 This is a schematic diagram illustrating how an image of interest is obtained from a head-mounted display screen using a user visual focus tracking system, as provided in an embodiment of this application. Figure 9 As shown, the overall image can be the screen of a head-mounted display. After the visual focus is determined by the eye-tracking module, the image of interest can be cropped with the visual focus as the center.

[0141] The technical solution provided in this application provides a method for efficiently acquiring users' personalized and spontaneous preferences for housing recommendations. By combining advanced technologies such as head-mounted displays, brain-computer interfaces, and artificial intelligence, it helps to improve the service level and user experience in the real estate market.

[0142] The system utilizes a brain-computer interface and eye-tracking module in a head-mounted display to achieve immersive viewing while accurately capturing EEG and eye movement signals, thus improving data quality. By combining EEG signals with deep learning models, it accurately perceives user emotional preferences, enabling personalized housing recommendations and increasing user satisfaction. Furthermore, by leveraging a visual Transformer model and a multimodal housing database, it rapidly extracts and matches features, effectively recommending properties that match user preferences and needs.

[0143] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0144] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0145] Figure 10 This is a schematic diagram of a house recommendation device provided in an embodiment of this application. Figure 10 As shown, the device includes:

[0146] Display module 1001 is configured to display information about a set of characteristic houses.

[0147] The visual feature extraction module 1002 is configured to track the visual focus features of a user when viewing a set of feature houses, and to determine the target feature image that the user is interested in.

[0148] The emotion feature extraction module 1003 is configured to acquire the emotion parameters of a user when viewing a target feature image, and determine the target feature image whose emotion parameters meet the preset recommendation conditions as the recommended feature image.

[0149] Recommendation module 1004 is configured to determine recommended houses based on recommendation feature images.

[0150] According to the technical solution provided in the embodiments of this application, by acquiring the visual focus features of a user when viewing a set of characteristic houses, the target feature image that the user is interested in is determined, and the emotional parameters of the user when viewing the target feature image are acquired. The target feature image whose emotional parameters meet the preset recommendation conditions is determined as the recommended feature image. Then, the recommended house is determined based on the recommended feature image, which can realize automatic house recommendation and improve recommendation efficiency. At the same time, by comprehensively utilizing the user's visual focus information and user emotions for house recommendation, the recommendation accuracy is improved.

[0151] In some implementations, tracking the visual focus characteristics of a user when viewing information about a set of characteristic houses to determine the target feature image that the user is interested in includes: determining the user's visual focus at a target time; cropping an image of a preset size from the image viewed by the user at the target time, centered on the visual focus, to obtain the target feature image; wherein the target time is any time when the user views the set of characteristic houses.

[0152] In some implementations, determining the user's visual focus at a target time includes: acquiring the user's eye rotation angle at the target time; determining the ray direction vector in the eye coordinate system based on the eye rotation angle; transforming the ray direction vector to the display coordinate system according to the calibration relationship between the eye coordinate system and the display coordinate system to obtain the line-of-sight equation in the display coordinate system; and determining the intersection of the line-of-sight equation and the plane of the display feature house set as the visual focus.

[0153] In some implementations, obtaining emotional parameters when a user views a target feature image includes: obtaining EEG data when the user views the target feature image; removing baseline data from the EEG data to obtain valid EEG data; and inputting the valid EEG data into a pre-trained emotion perception model to obtain emotional parameters.

[0154] In some implementations, the pre-trained emotion perception model includes a multi-scale temporal feature extraction network, a self-attention feature extraction network, and an emotion estimation network. The multi-scale temporal feature extraction network includes multiple convolutional layers and a multi-scale fusion module. The multiple convolutional layers extract features from the effective EEG data respectively, and the multi-scale fusion module fuses the features extracted by each convolutional layer to obtain multi-scale features. The self-attention feature extraction network generates an attention feature matrix based on the multi-scale features. The attention feature matrix is ​​used to characterize the relationship between different time steps and different electrodes acquiring EEG data. The emotion estimation network maps the attention feature matrix into a two-dimensional vector, processes the two-dimensional vector using a growth curve function to obtain the preferred arousal value, and processes the two-dimensional vector using a hyperbolic tangent function to obtain the preferred efficacy value. The emotion parameters include the preferred arousal value and the preferred efficacy value.

[0155] In some implementations, the self-attention feature extraction network generates an attention feature matrix based on multi-scale features, including: determining the feature dimension and number of sequences of the multi-scale features, where the number of sequences is the product of the number of electrodes and the number of time steps in the collected EEG data; embedding and encoding the multi-scale features based on the feature dimension and the number of sequences to obtain an input feature matrix; adding positional encoding to the input feature matrix; and using a multi-head self-attention mechanism to extract features from the input feature matrix with added positional encoding to obtain the attention feature matrix.

[0156] In some implementations, determining that the emotional parameters meet preset recommendation conditions includes: obtaining the preference arousal value and preference effectiveness value in the emotional parameters; determining that the preference arousal value is greater than a first preset threshold and the preference effectiveness value is greater than a second preset threshold, and thus determining that the emotional parameters meet the preset recommendation conditions.

[0157] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0158] Figure 11 This is a schematic diagram of the electronic device provided in an embodiment of this application. For example... Figure 11As shown, the electronic device 11 of this embodiment includes: a processor 1101, a memory 1102, and a computer program 1103 stored in the memory 1102 and executable on the processor 1101. When the processor 1101 executes the computer program 1103, it implements the steps in the various method embodiments described above. Alternatively, when the processor 1101 executes the computer program 1103, it implements the functions of each module / unit in the various device embodiments described above.

[0159] Electronic device 11 may be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 11 may include, but is not limited to, processor 1101 and memory 1102. Those skilled in the art will understand that... Figure 11 This is merely an example of electronic device 11 and does not constitute a limitation on electronic device 11. It may include more or fewer components than shown, or different components.

[0160] The processor 1101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0161] The memory 1102 can be an internal storage unit of the electronic device 11, such as a hard disk or RAM of the electronic device 11. The memory 1102 can also be an external storage device of the electronic device 11, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, FlashCard, etc., equipped on the electronic device 11. The memory 1102 can also include both internal and external storage units of the electronic device 11. The memory 1102 is used to store computer programs and other programs and data required by the electronic device.

[0162] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0163] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0164] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A housing recommendation method characterized by comprising: include: Display information on characteristic housing sets; Track the visual focus characteristics of users when viewing the set of characteristic houses to determine the target feature images that the users are interested in; Acquire the emotional parameters of the user when viewing the target feature image, and determine the target feature image whose emotional parameters meet the preset recommendation conditions as the recommended feature image; Recommended houses are determined based on the recommended feature images; The acquisition of the emotional parameters of the user when viewing the target feature image includes: Acquire electroencephalogram (EEG) data of the user while viewing the target feature image; Baseline data is removed from the EEG data to obtain valid EEG data; The effective EEG data is input into a pre-trained emotion perception model to obtain the emotion parameters. The pre-trained emotion perception model includes a multi-scale temporal feature extraction network, a self-attention feature extraction network, and an emotion estimation network. The multi-scale temporal feature extraction network includes multiple convolutional layers and a multi-scale fusion module. The multiple convolutional layers extract features from the effective EEG data, and the multi-scale fusion module fuses the features extracted by each convolutional layer to obtain multi-scale features. The self-attention feature extraction network generates an attention feature matrix based on the multi-scale features. The attention feature matrix is ​​used to characterize the relationship between different time steps and different electrodes acquiring the EEG data. The emotion estimation network maps the attention feature matrix into a two-dimensional vector, processes the two-dimensional vector using a growth curve function to obtain a preferred arousal value, and processes the two-dimensional vector using a hyperbolic tangent function to obtain a preferred efficacy value. The emotion parameters include the preferred arousal value and the preferred efficacy value. The determination that the emotion parameters meet the preset recommendation conditions includes: Obtain the preference arousal value and preference efficacy value from the aforementioned emotion parameters; If the preference arousal value is determined to be greater than a first preset threshold and the preference efficacy value is determined to be greater than a second preset threshold, then the emotion parameter is determined to meet the preset recommendation conditions.

2. The method of claim 1, wherein, Tracking the visual focus characteristics of users when viewing the set of featured houses, and determining the target feature images of interest to the user, including: Determine the user's visual focus at the target time; From the image viewed by the user at the target time, a preset size image is cropped with the visual focus as the center to obtain the target feature image; The target time is any time when the user views the information of the feature house set.

3. The method of claim 2, wherein, Determine the user's visual focus at the target time, including: Obtain the user's eye rotation angle at the target time; The ray direction vector in the eye coordinate system is determined based on the eye rotation angle. Based on the calibration relationship between the eye coordinate system and the display coordinate system, the ray direction vector is transformed to the display coordinate system to obtain the line-of-sight equation of the display coordinate system; The intersection of the line-of-sight equation and the plane displaying the information of the feature house set is determined as the visual focus.

4. The method of claim 1, wherein, The self-attention feature extraction network generates an attention feature matrix based on the multi-scale features, including: Determine the feature dimension and number of sequences of the multi-scale features, wherein the number of sequences is the product of the number of electrodes and the number of time steps used to collect the EEG data; The multi-scale features are embedded and encoded based on the feature dimension and the number of sequences to obtain the input feature matrix; Add positional encoding to the input feature matrix; The attention feature matrix is ​​obtained by extracting features from the input feature matrix with added positional encoding using a multi-head self-attention mechanism.

5. A house recommendation device for performing the method according to any one of claims 1-4, characterized in that, include: The display module is configured to display information about a set of characteristic houses; The visual feature extraction module is configured to track the visual focus features of a user when viewing the feature house set information, and determine the target feature image that the user is interested in; The emotion feature extraction module is configured to acquire the emotion parameters of a user when viewing the target feature image, and determine the target feature image whose emotion parameters meet the preset recommendation conditions as the recommended feature image. The recommendation module is configured to determine recommended houses based on the recommendation feature image.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

Images