User social information recommendation method and device based on multi-modal data, medium and equipment

Abstract

The invention provides a user social information recommendation method and device based on multi-modal data, a medium and equipment, and belongs to the technical field of social applications. The method comprises the following steps: in response to a friend-making pairing instruction input by a first campus user, acquiring multi-modal data of the first campus user; according to the multi-modal data, constructing a first behavior trajectory diagram of the first campus user; in a preset campus user database, determining a second behavior track diagram matched with the first behavior track diagram and a second campus user corresponding to the second behavior track diagram; under the condition that selection instructions of the first campus user and the second campus user on a second place identifier of the second interaction place are received, place recommendation configuration of the first campus user is determined according to place configuration of the second interaction place; and sending the place recommendation configuration to the first campus user.

Description

Technical Field

[0001] This invention relates to the technical field of social applications, and more specifically, to a method, apparatus, medium, and device for recommending user social information based on multimodal data. Background Technology

[0002] With the rapid development of social applications, they can be widely used within various groups such as hospitals, businesses, and university campuses. Currently, existing social applications targeting university campuses typically require real-name authentication and create communities based on users' classes, departments, grades, or clubs. Users then connect through chat or comments within these communities. However, this approach is not user-friendly for students who are not good at expressing themselves or have heavy academic workloads, resulting in a poor user experience. Summary of the Invention

[0003] To address the aforementioned problems, the present invention aims to provide a method, apparatus, storage medium, and electronic device for recommending user social information based on multimodal data.

[0004] In a first aspect, embodiments of the present invention provide a method for recommending user social information based on multimodal data, including: In response to a dating and matching instruction input by a first campus user, multimodal data of the first campus user is obtained; wherein, the multimodal data includes location identifiers representing the interaction locations of the first campus user within the campus area and the interaction times corresponding to the location identifiers; Based on the multimodal data, a first behavioral trajectory map of the first campus user is constructed; In a pre-set campus user database, a second behavior trajectory map matching the first behavior trajectory map and a second campus user corresponding to the second behavior trajectory map are determined; wherein, the first behavior trajectory map and the second behavior trajectory map have a first location identifier of a first interaction location; Upon receiving selection instructions from the first campus user and the second campus user regarding the second location identifier of the second interaction location, the recommended location configuration for the first campus user is determined based on the location configuration of the second interaction location. Send the recommended location configuration to the first campus user.

[0005] In some optional implementations, the interaction locations include a cafeteria, a library, and a public classroom. The multimodal data includes first multimodal data, second multimodal data, and third multimodal data. The first multimodal data includes a first location identifier of the first campus user in the target cafeteria, a first interaction time corresponding to the first location identifier, and a first interaction behavior. The second multimodal data includes a second location identifier of the first campus user in the target library, a second interaction time corresponding to the second location identifier, and a second interaction behavior. The third multimodal data includes a third location identifier of the first campus user in the target public classroom, a third interaction time corresponding to the third location identifier, and a third interaction behavior.

[0006] In some optional implementations, constructing the first behavioral trajectory map of the first campus user based on the multimodal data includes: Determine the first time period belonging to the target time node among the first interaction time, the second interaction time, and the third interaction time; Based on the first location identifier, the second location identifier, and the third location identifier corresponding to the first time period, a movement path of the first campus user at the target time node is constructed; wherein, the first path node of the movement path corresponds to the first location identifier, the second path node of the movement path corresponds to the second location identifier, and the third path node of the movement path corresponds to the third location identifier. A first behavior trajectory map is constructed based on the movement path, the first interaction behavior corresponding to the first location identifier, the second interaction behavior corresponding to the second location identifier, and the third interaction behavior corresponding to the third location identifier.

[0007] In some optional implementations, determining, in a pre-set campus user database, a second behavioral trajectory map matching the first behavioral trajectory map and the second campus user corresponding to the second behavioral trajectory map includes: A preset image processing network model is invoked to determine a second behavioral trajectory map and a second campus user corresponding to the second behavioral trajectory map in a preset campus user database that meet the set matching conditions. The matching conditions include that, at the target time node, the second behavioral trajectory map has at least one of a first interaction event, a second interaction event, and a third interaction event. The first interaction event is associated with the first location identifier and the first interaction behavior, the second interaction event is associated with the second location identifier and the second interaction behavior, and the third interaction event is associated with the third location identifier and the third interaction behavior.

[0008] In some optional implementations, before invoking a preset image processing network model to determine the second behavioral trajectory map that meets the set matching conditions and the second campus user corresponding to the second behavioral trajectory map, the method further includes: Obtain the preferred user attributes of the first campus user; In a pre-set campus user database, identify campus users who match the preferred user attributes and the corresponding user data set for those campus users; The step of calling a preset image processing network model to determine the second behavioral trajectory map that meets the set matching conditions in the preset campus user database and the second campus user corresponding to the second behavioral trajectory map includes: A preset image processing network model is invoked to determine the second behavioral trajectory map that meets the set matching conditions in the user data set, as well as the second campus user corresponding to the second behavioral trajectory map.

[0009] In some optional implementations, the method further includes: Send an input window to the first campus user and the second campus user, in which the first campus user sets the user preference ratios for the first interaction event, the second interaction event, and the third interaction event; The system obtains the first user preference ratio and the first expected time output by the first campus user in the input window, and the second user preference ratio and the second expected time output by the second campus user in the input window, as selection indications for the second location identifier of the second interaction location by the first campus user and the second campus user.

[0010] In some optional implementations, determining the recommended location configuration for the first campus user based on the location configuration of the second interaction location includes: When the second interaction location corresponds to the first location identifier, the location area and the first configuration time near the first interaction object indicated by the first interaction behavior are determined according to the location configuration of the second interaction scenario, so as to serve as the recommended location configuration for the first campus user. When the second interactive location corresponds to the second location identifier or the third location identifier, the location area where the first interactive object indicated by the second interactive behavior is located and the location seat of the location area in the second configuration time are determined according to the location configuration of the second interactive scenario, so as to serve as the location recommendation configuration for the first campus user.

[0011] Secondly, embodiments of the present invention also provide a user social information recommendation device based on multimodal data, comprising: The response module is used to respond to the dating and matching instruction input by the first campus user and obtain the multimodal data of the first campus user; wherein, the multimodal data includes a location identifier representing the interaction location of the first campus user within the campus area and the interaction time corresponding to the location identifier; The construction module is used to construct the first behavioral trajectory map of the first campus user based on the multimodal data; The user identification module is used to identify, in a preset campus user database, a second behavior trajectory map that matches the first behavior trajectory map and a second campus user corresponding to the second behavior trajectory map; wherein, the first behavior trajectory map and the second behavior trajectory map have a first location identifier of a first interaction location; The configuration determination module is used to determine the recommended location configuration for the first campus user based on the location configuration of the second interaction location when receiving selection instructions from the first campus user and the second campus user for the second location identifier of the second interaction location. The sending module is used to send the recommended location configuration to the first campus user.

[0012] Thirdly, embodiments of the present invention also provide a computer storage medium storing computer-executable instructions for use in any of the above-described user social information recommendation methods based on multimodal data.

[0013] Fourthly, embodiments of the present invention also provide an electronic device, comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform any of the above-described user social information recommendation methods based on multimodal data.

[0014] In the solution provided by the first aspect of this invention, a first campus user can send a friend matching instruction, and the server can obtain the first campus user's multimodal data and construct a first behavioral trajectory map of the first campus user using the multimodal data. Then, a second behavioral trajectory map matching the first behavioral trajectory map and the second campus user corresponding to the second behavioral trajectory map are determined. When the first and second campus users select a second location identifier for the second interaction location, the first campus user is set to use a location recommendation configuration to make friends with the second campus user in the second interaction location. This eliminates the need for campus users to actively make friends, making it fast, convenient, and providing a better user experience.

[0015] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 The flowchart illustrates a user social information recommendation method based on multimodal data provided by an embodiment of the present invention. Figure 2 This diagram illustrates the structure of a user social information recommendation device based on multimodal data, as provided in an embodiment of the present invention. Figure 3 A schematic diagram of the structure of an electronic device for performing a user social information recommendation method based on multimodal data, provided by an embodiment of the present invention, is shown. Detailed Implementation

[0018] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0019] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0020] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0021] This invention provides a method for recommending user social information based on multimodal data. See also: Figure 1 As shown, the execution entity of this method can be a server within the campus, which, with the authorization of campus users, can obtain multimodal data from various campus users. The method includes: Step 101: In response to the dating and matching instruction input by the first campus user, obtain the multimodal data of the first campus user; wherein, the multimodal data includes the location identifiers representing the interaction locations of the first campus user within the campus area and the interaction times corresponding to the location identifiers.

[0022] In this embodiment, campus users can be students, teachers, or administrators who have configured mobile terminals. The first campus user can be a specific campus user. The first campus user can select a matchmaking option on the matchmaking interface of the dating platform associated with the server. The server can obtain the multimodal data of the first campus user, provided that the first campus user authorizes the data.

[0023] In this embodiment, the interactive location is, for example, a library, a canteen, or a public classroom. The campus may have multiple libraries, canteens, and public classrooms, each with a different location identifier. For example, the location identifier for the East Library is LI01, the West Canteen is RE02, and the classroom in Building 2 of the Art Building is RO2. The interaction time can be the registration operation performed by the first campus user at the interactive location via a mobile terminal. For example, if the interactive location is the East Library, and the first campus user uses a mobile terminal to borrow items at the East Library's borrowing terminal at 9:20 AM on January 6th, then the location identifier is LI01, and the interaction time is 01.06.09:20.

[0024] Step 102: Based on the multimodal data, construct the first behavioral trajectory map of the first campus user.

[0025] In some embodiments, the interaction locations include a cafeteria, a library, and a public classroom. The multimodal data includes first multimodal data, second multimodal data, and third multimodal data. The first multimodal data includes a first location identifier of a first campus user in the target cafeteria, a first interaction time corresponding to the first location identifier, and a first interaction behavior. The second multimodal data includes a second location identifier of a first campus user in the target library, a second interaction time corresponding to the second location identifier, and a second interaction behavior. The third multimodal data includes a third location identifier of a first campus user in the target public classroom, a third interaction time corresponding to the third location identifier, and a third interaction behavior.

[0026] In some instances, the first multimodal data is a transaction made by the first campus user at window 4 of the West Cafeteria at 11:30 AM on January 6th. The target cafeteria is the West Cafeteria, the first location is identified as RE02, the first interaction time is 01.06.11:30, and the first interaction behavior is W04.

[0027] In some instances, the second multimodal data is the first campus user borrowing a book of type TL01 from the East Library at 9:20 AM on January 6th. The target library is the East Library, the second location identifier is LI01, the second interaction time is 01.06.9:20, and the second interaction behavior is TL01.

[0028] In some instances, the third multimodal data is that the first campus user reserved seat 0201 in classroom 2 of the Art Building on January 6th from 7 PM to 10 PM. The target public classroom is classroom 2 of the Art Building, the third location is identified as RO2, the third interaction time is 01.06.19-22, and the second interaction behavior is 0201.

[0029] In this embodiment, by digitizing multimodal data, the storage efficiency of multimodal data can be effectively improved, and the behavioral profiles of campus users can be accurately depicted to achieve subsequent recommendation and matching of second campus users.

[0030] Specifically, step 102 above, "Constructing the first behavioral trajectory map of the first campus user based on multimodal data", includes steps A1 to A3.

[0031] Step A1: Determine the first time segment belonging to the target time node among the first interaction time, the second interaction time, and the third interaction time.

[0032] In this embodiment, the first interaction time is 01.06.11:30, the second interaction time is 01.06.9:20, and the third interaction time is 01.06.19-22. All three times fall under the category of Tuesday, January 6th. The first interaction time falls within the noon time slot of the target time node, thus defining it as the first time slot of the first interaction time belonging to the target node. The second interaction time falls within the morning time slot of the target time node, thus defining it as the first time slot of the second interaction time belonging to the target node. The third interaction time falls within the evening time slot of the target time node, thus defining it as the first time slot of the third interaction time belonging to the target node.

[0033] Step A2: Based on the first location identifier, second location identifier, and third location identifier corresponding to the first time period, construct the movement path of the first campus user at the target time node; wherein, the first path node of the movement path corresponds to the first location identifier, the second path node of the movement path corresponds to the second location identifier, and the third path node of the movement path corresponds to the third location identifier.

[0034] In some instances, the location coordinates of RE02, LI01, and RO2 on the campus map can be determined using the first, second, and third location identifiers. Then, combining this with the data from 01.06.11:30, 01.06.9:20, and 01.06.19-22, the movement path of the first campus user on January 6th can be obtained as amLI01-noonRE02-pmRO2. The first path node corresponds to the location coordinates of RE02 on the campus map, the second path node corresponds to the location coordinates of LI01 on the campus map, and the third path node corresponds to the location coordinates of RO2 on the campus map.

[0035] Step A3: Construct the first behavior trajectory map based on the movement path, the first interaction behavior corresponding to the first location identifier, the second interaction behavior corresponding to the second location identifier, and the third interaction behavior corresponding to the third location identifier.

[0036] In this embodiment, corresponding interactive behavior labels are set for each path node in the movement path. That is, the location coordinates of RE02 on the campus map are set with the label W04, the location coordinates of LI01 on the campus map are set with the label TL01, and the location coordinates of RO2 on the campus map are set with the label 0201. The movement path with the labels set is used as the first behavior trajectory map constructed.

[0037] In this embodiment, a first behavioral trajectory map is constructed using multimodal data to provide technical support for subsequent image processing network models to identify features of the first behavioral trajectory map.

[0038] Step 103: In the pre-set campus user database, determine the second behavior trajectory map that matches the first behavior trajectory map and the second campus user corresponding to the second behavior trajectory map; wherein, the first behavior trajectory map and the second behavior trajectory map have a first location identifier of the first interaction location.

[0039] Specifically, step 102 above, "in the preset campus user database, determine the second behavior trajectory map that matches the first behavior trajectory map and the second campus user corresponding to the second behavior trajectory map", includes step B1.

[0040] Step B1: Invoke the preset image processing network model to determine the second behavioral trajectory map and the second campus user corresponding to the second behavioral trajectory map in the preset campus user database that meet the set matching conditions; wherein, the matching conditions include that the second behavioral trajectory map has at least one of the first interaction event, the second interaction event and the third interaction event at the target time node; wherein, the first interaction event is associated with the first location identifier and the first interaction behavior, the second interaction event is associated with the second location identifier and the second interaction behavior, and the third interaction event is associated with the third location identifier and the third interaction behavior.

[0041] In this embodiment, the image processing network model is an existing deep learning network model. This image processing network model can identify various behavioral trajectory maps through image recognition to determine a second behavioral trajectory map that is the same as or similar to the first behavioral trajectory map. Determining the second behavioral trajectory map that matches the first behavioral trajectory map using the image processing network model is prior art and will not be described in detail here.

[0042] Specifically, before step B1, "calling a preset image processing network model to determine the second behavioral trajectory map and the second campus user corresponding to the second behavioral trajectory map in the preset campus user database that meet the set matching conditions", the method also includes steps B2 and B3.

[0043] Step B2: Obtain the preferred user attributes of the first campus user.

[0044] In this embodiment, the preferred user attributes are, for example, age restrictions, grade restrictions, or major restrictions.

[0045] Step B3: In the pre-set campus user database, identify campus users who match the preferred user attributes and the corresponding user data set for each campus user.

[0046] In this embodiment, the preferred user attributes are 25 years old, senior, and majoring in computer science. The campus user database can retrieve all campus users who match this preferred user attribute.

[0047] Based on this, step B1, "calling a preset image processing network model to determine the second behavioral trajectory map and the second campus user corresponding to the second behavioral trajectory map in the preset campus user database that meet the set matching conditions," includes step B4: Step B4: Call the preset image processing network model to determine the second behavioral trajectory map that meets the set matching conditions in the user data set, and the second campus user corresponding to the second behavioral trajectory map.

[0048] In this embodiment, by adding preferred user attributes to filter second campus users, the accuracy of first campus users in filtering second campus users that meet their needs can be further improved.

[0049] Step 104: Upon receiving the selection instructions from the first campus user and the second campus user for the second location identifier of the second interaction location, determine the recommended location configuration for the first campus user based on the location configuration of the second interaction location.

[0050] Specifically, before step 104, "upon receiving the selection instruction of the first campus user and the second campus user for the second location identifier of the second interaction location, determining the recommended location configuration for the first campus user based on the location configuration of the second interaction location," the method further includes steps C1 and C2: Step C1: Send an input window to the first campus user and the second campus user, in which the first campus user sets the user preference ratios for the first, second, and third interactive events.

[0051] Step C2: Obtain the first user preference ratio and first expected time output by the first campus user in the input window, and obtain the second user preference ratio and second expected time output by the second campus user in the input window, as an indication for the first campus user and the second campus user to select the second location identifier of the second interaction location.

[0052] In some instances, the input window can be an input box. A first campus user can input 40% West Cafeteria, 20% East Library, and 40% Art Building 2; a second campus user can input 50% West Cafeteria, 0% East Library, and 50% Art Building 2. The second interaction location can be randomly selected from the West Cafeteria and Art Building 2. The first campus user can input a first desired time, and the second campus user can input a second desired time. The server can obtain the overlap between the first and second desired times to use as the agreed time in the recommended location configuration.

[0053] In this embodiment, by having campus users select an interactive location as a place for making friends with strangers, the occurrence of campus users directly expressing their intentions can be reduced.

[0054] Specifically, step 101 above, "determine the recommended location configuration for the first campus user based on the location configuration of the second interaction location," includes steps D1 and D2.

[0055] Step D1: When the second interaction location corresponds to the first location identifier, determine the location area and the first configuration time near the first interaction object indicated by the first interaction behavior according to the location configuration of the second interaction scenario, so as to serve as the recommended location configuration for the first campus user.

[0056] In some instances, the first configuration time can be the overlap between the first expected time and the second expected time. The first interactive behavior is W04. The server can determine the location area corresponding to W04 through a preset first correspondence, for example, W04 corresponds to area B1. This first correspondence reflects that different windows correspond to different location areas.

[0057] Step D2: If the second interaction location corresponds to the second location identifier or the third location identifier, determine the location area where the first interaction object indicated by the second interaction behavior is located and the location seat of the location area in the second configuration time according to the location configuration of the second interaction scenario, so as to serve as the location recommendation configuration for the first campus user.

[0058] In some instances, the server can determine the location area corresponding to the second interactive behavior through a pre-defined second correspondence. The server then retrieves remaining seats in that area that were not booked during the second configured time and identifies two adjacent seats within that area. Similarly, the server can determine the location area corresponding to the third interactive behavior through a pre-defined third correspondence. The server then retrieves remaining seats in that area that were not booked during the second configured time and identifies two adjacent seats within that area. Here, the second correspondence reflects that different library seats correspond to different location areas. The third correspondence reflects that different classroom seats correspond to different location areas.

[0059] In this embodiment, by configuring location settings for campus users, it is possible to enable matching and making friends among campus users.

[0060] Step 105: Send the recommended location configuration to the first campus user.

[0061] In this embodiment, after the first campus user receives the location recommendation configuration, they can go to the designated second interactive location through the location recommendation configuration.

[0062] This invention provides a user social information recommendation method based on multimodal data. A first campus user can send a friend matching request. The server can acquire the first campus user's multimodal data and construct a first behavioral trajectory map of the first campus user using this data. Then, a second behavioral trajectory map matching the first behavioral trajectory map and the corresponding second campus user are determined. Based on the selection instructions of the first and second campus users regarding a second location identifier for a second interaction location, the first campus user is configured to make friends with the second campus user in the second interaction location using a recommended location setting. This method eliminates the need for campus users to actively make friends, is fast and convenient, and provides a better user experience.

[0063] The above describes in detail the process of recommending user social information based on multimodal data. This method can also be implemented using a corresponding device, the structure and function of which will be described in detail below.

[0064] Based on the same inventive concept, embodiments of the present invention also provide a user social information recommendation device based on multimodal data, see [link to related document]. Figure 2 As shown, the device includes: The response module 201 is used to respond to the dating and matching instruction input by the first campus user and obtain the multimodal data of the first campus user; wherein, the multimodal data includes the location identifier representing the interaction location of the first campus user within the campus area and the interaction time corresponding to the location identifier; Module 202 is used to construct the first behavioral trajectory map of the first campus user based on multimodal data; User determination module 203 is used to determine, in a preset campus user database, a second behavior trajectory map that matches the first behavior trajectory map and a second campus user corresponding to the second behavior trajectory map; wherein, the first behavior trajectory map and the second behavior trajectory map have a first location identifier of a first interaction location; The configuration determination module 204 is used to determine the recommended location configuration for the first campus user based on the location configuration of the second interaction location when receiving the selection instruction of the second location identifier of the second interaction location from the first campus user and the second campus user. The sending module 205 is used to send the location recommendation configuration to the first campus user.

[0065] In some optional implementations, the construction module 202 is further configured to determine a first time period belonging to the target time node among the first interaction time, the second interaction time, and the third interaction time; construct the movement path of the first campus user at the target time node based on the first location identifier, the second location identifier, and the third location identifier corresponding to the first time period; wherein, the first path node of the movement path corresponds to the first location identifier, the second path node of the movement path corresponds to the second location identifier, and the third path node of the movement path corresponds to the third location identifier; and construct a first behavior trajectory map based on the movement path, the first interaction behavior corresponding to the first location identifier, the second interaction behavior corresponding to the second location identifier, and the third interaction behavior corresponding to the third location identifier.

[0066] In some optional implementations, the user determination module 203 is further configured to invoke a preset image processing network model to determine a second behavioral trajectory map and a second campus user corresponding to the second behavioral trajectory map in a preset campus user database that meet the set matching conditions; wherein, the matching conditions include that the second behavioral trajectory map has at least one of a first interaction event, a second interaction event, and a third interaction event at a target time node; wherein, the first interaction event is associated with a first location identifier and a first interaction behavior, the second interaction event is associated with a second location identifier and a second interaction behavior, and the third interaction event is associated with a third location identifier and a third interaction behavior.

[0067] In some optional implementations, the device further includes a set acquisition module for acquiring the preferred user attributes of the first campus user; and in a preset campus user database, determining the campus users that match the preferred user attributes and the user data set corresponding to the campus users.

[0068] The user identification module 203 is also used to call a preset image processing network model to identify the second behavioral trajectory map that meets the set matching conditions in the user data set and the second campus user corresponding to the second behavioral trajectory map.

[0069] In some optional embodiments, the device further includes a window sending module for sending an input window to a first campus user and a second campus user, wherein the first campus user sets the user preference ratios for a first interaction event, a second interaction event, and a third interaction event; acquiring the first user preference ratio and a first desired time output by the first campus user in the input window, and acquiring the second user preference ratio and the second desired time output by the second campus user in the input window, as selection instructions for the second location identifier of the second interaction location by the first campus user and the second campus user.

[0070] In some optional implementations, the configuration determination module 204 is further configured to, when the second interaction location corresponds to the first location identifier, determine the location area near the first interactive object indicated by the first interactive behavior and the first configuration time according to the location configuration of the second interaction scenario, so as to serve as the recommended location configuration for the first campus user; when the second interaction location corresponds to the second location identifier or the third location identifier, determine the location area where the first interactive object indicated by the second interactive behavior is located and the location seat of the location area at the second configuration time according to the location configuration of the second interaction scenario, so as to serve as the recommended location configuration for the first campus user.

[0071] This invention also provides a computer storage medium storing computer-executable instructions, including a program for executing the above-described user social information recommendation method based on multimodal data. The computer-executable instructions can execute the methods in any of the above-described method embodiments.

[0072] The computer storage medium can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical storage (e.g., CD, DVD, BD, HVD), and semiconductor storage (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0073] Figure 3 A structural block diagram of an electronic device according to another embodiment of the present invention is shown. The electronic device 1100 may be a host server with computing capabilities, a personal computer (PC), or a portable computer or terminal, etc. The specific embodiments of the present invention do not limit the specific implementation of the electronic device.

[0074] The electronic device 1100 includes at least one processor 1110, a communications interface 1120, a memory array 1130, and a bus 1140. The processor 1110, the communications interface 1120, and the memory 1130 communicate with each other via the bus 1140.

[0075] The communication interface 1120 is used to communicate with network elements, including, for example, virtual machine management centers and shared storage.

[0076] Processor 1110 is used to execute programs. Processor 1110 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0077] Memory 1130 is used for executable instructions. Memory 1130 may include high-speed RAM memory and may also include non-volatile memory, such as at least one disk storage device. Memory 1130 may also be a memory array. Memory 1130 may also be divided into blocks, and the blocks may be combined into virtual volumes according to certain rules. The instructions stored in memory 1130 can be executed by processor 1110 to enable processor 1110 to execute the user social information recommendation method based on multimodal data in any of the above method embodiments.

[0078] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for recommending user social information based on multimodal data, characterized in that, include: In response to a dating and matching instruction input by a first campus user, multimodal data of the first campus user is obtained; wherein, the multimodal data includes location identifiers representing the interaction locations of the first campus user within the campus area and the interaction times corresponding to the location identifiers; Based on the multimodal data, a first behavioral trajectory map of the first campus user is constructed; In a pre-set campus user database, a second behavior trajectory map matching the first behavior trajectory map and a second campus user corresponding to the second behavior trajectory map are determined; wherein, the first behavior trajectory map and the second behavior trajectory map have a first location identifier of a first interaction location; Upon receiving selection instructions from the first campus user and the second campus user regarding the second location identifier of the second interaction location, the recommended location configuration for the first campus user is determined based on the location configuration of the second interaction location. Send the recommended location configuration to the first campus user.

2. The method according to claim 1, characterized in that, The interaction locations include a cafeteria, a library, and public classrooms. The multimodal data includes first multimodal data, second multimodal data, and third multimodal data. The first multimodal data includes a first location identifier of the first campus user in the target cafeteria, a first interaction time corresponding to the first location identifier, and a first interaction behavior. The second multimodal data includes a second location identifier of the first campus user in the target library, a second interaction time corresponding to the second location identifier, and a second interaction behavior. The third multimodal data includes a third location identifier of the first campus user in the target public classroom, a third interaction time corresponding to the third location identifier, and a third interaction behavior.

3. The method according to claim 2, characterized in that, The step of constructing a first behavioral trajectory map of the first campus user based on the multimodal data includes: Determine the first time period belonging to the target time node among the first interaction time, the second interaction time, and the third interaction time; Based on the first location identifier, the second location identifier, and the third location identifier corresponding to the first time period, a movement path of the first campus user at the target time node is constructed; wherein, the first path node of the movement path corresponds to the first location identifier, the second path node of the movement path corresponds to the second location identifier, and the third path node of the movement path corresponds to the third location identifier. A first behavior trajectory map is constructed based on the movement path, the first interaction behavior corresponding to the first location identifier, the second interaction behavior corresponding to the second location identifier, and the third interaction behavior corresponding to the third location identifier.

4. The method according to claim 3, characterized in that, The step of determining, in a pre-set campus user database, a second behavior trajectory map matching the first behavior trajectory map and the second campus user corresponding to the second behavior trajectory map, includes: A preset image processing network model is invoked to determine a second behavioral trajectory map and a second campus user corresponding to the second behavioral trajectory map in a preset campus user database that meet the set matching conditions. The matching conditions include that, at the target time node, the second behavioral trajectory map has at least one of a first interaction event, a second interaction event, and a third interaction event. The first interaction event is associated with the first location identifier and the first interaction behavior, the second interaction event is associated with the second location identifier and the second interaction behavior, and the third interaction event is associated with the third location identifier and the third interaction behavior.

5. The method according to claim 3, characterized in that, Before calling a preset image processing network model to determine the second behavioral trajectory map that meets the set matching conditions and the second campus user corresponding to the second behavioral trajectory map, the method further includes: Obtain the preferred user attributes of the first campus user; In a pre-set campus user database, identify campus users who match the preferred user attributes and the corresponding user data set for those campus users; The step of calling a preset image processing network model to determine the second behavioral trajectory map that meets the set matching conditions in the preset campus user database and the second campus user corresponding to the second behavioral trajectory map includes: A preset image processing network model is invoked to determine the second behavioral trajectory map that meets the set matching conditions in the user data set, as well as the second campus user corresponding to the second behavioral trajectory map.

6. The method according to claim 3, characterized in that, The method further includes: Send an input window to the first campus user and the second campus user, in which the first campus user sets the user preference ratios for the first interaction event, the second interaction event, and the third interaction event; The system obtains the first user preference ratio and the first expected time output by the first campus user in the input window, and the second user preference ratio and the second expected time output by the second campus user in the input window, as selection indications for the second location identifier of the second interaction location by the first campus user and the second campus user.

7. The method according to claim 1, characterized in that, The step of determining the recommended location configuration for the first campus user based on the location configuration of the second interaction location includes: When the second interaction location corresponds to the first location identifier, the location area and the first configuration time near the first interaction object indicated by the first interaction behavior are determined according to the location configuration of the second interaction scenario, so as to serve as the recommended location configuration for the first campus user. When the second interactive location corresponds to the second location identifier or the third location identifier, the location area where the first interactive object indicated by the second interactive behavior is located and the location seat of the location area in the second configuration time are determined according to the location configuration of the second interactive scenario, so as to serve as the location recommendation configuration for the first campus user.

8. A user social information recommendation device based on multimodal data, characterized in that, include: The response module is used to respond to the dating and matching instruction input by the first campus user and obtain the multimodal data of the first campus user; wherein, the multimodal data includes a location identifier representing the interaction location of the first campus user within the campus area and the interaction time corresponding to the location identifier; A construction module is used to construct a first behavioral trajectory map of the first campus user based on the multimodal data; The user identification module is used to identify, in a preset campus user database, a second behavior trajectory map that matches the first behavior trajectory map and a second campus user corresponding to the second behavior trajectory map; wherein, the first behavior trajectory map and the second behavior trajectory map have a first location identifier of a first interaction location; The configuration determination module is used to determine the recommended location configuration for the first campus user based on the location configuration of the second interaction location when receiving selection instructions from the first campus user and the second campus user for the second location identifier of the second interaction location. The sending module is used to send the recommended location configuration to the first campus user.

9. A computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions for executing the user social information recommendation method based on multimodal data as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the user social information recommendation method based on multimodal data as described in any one of claims 1 to 7.

Images