Information recommendation method and system based on object-related data, and storage medium

Abstract

The application provides an information recommendation method and system based on object-related data, and a storage medium, the method comprising: when a first terminal and a second terminal are located in a same target area range, the first terminal sends notification information of entering a target mode to the second terminal in response to a target mode selection instruction triggered based on a mode selection interface, and collects target feature data related to the target mode in response to an object feature data collection instruction, and sends the target feature data to a server; the second terminal sends a corresponding playing request to the server; the server analyzes the target feature data based on object-related data analysis to obtain a target-related data analysis result, determines playing information corresponding to the target-related data analysis result, and sends the playing information to the second terminal; and the second terminal plays the playing information. The application relates to natural language processing technology in artificial intelligence, and can realize accurate information recommendation based on object-related data.

Description

[0001] The application is a divisional application of the application with the application number 2020107623352, the application date of 2020.07.31, and the application name of a information recommendation method, system and storage medium based on user behavior. TECHNICAL FIELD

[0002] The application belongs to the technical field of smart home, and particularly relates to a information recommendation method, system and storage medium based on object association data. BACKGROUND

[0003] Smart home is a manifestation of materialization under the influence of the Internet. Smart home connects various smart devices (such as audio and video devices, lighting systems, curtain control, air conditioning control, digital theater systems, video servers, video cabinet systems, network home appliances, etc.) in the home together through the Internet of Things technology, and provides home appliance control, lighting control, telephone remote control, indoor and outdoor remote control, environment monitoring, heating control, infrared conversion, and programmable timing control, etc. Compared with ordinary home, smart home not only has traditional living functions, but also combines architecture, network communication, information home appliances, and device automation, and provides full-range information interaction functions, and even saves money for various energy costs.

[0004] With the continuous development of the Internet of Things industry, the concept of smart home is gradually accepted by people, and people's demand for smart home is increasing day by day. How to realize accurate information recommendation based on object association data through the linkage of each smart device in the physical network is particularly important. SUMMARY

[0005] In order to realize accurate information recommendation based on object association data through the linkage of each smart device in the Internet of Things in the smart home application scene, the application provides a information recommendation method, system and storage medium based on object association data.

[0006] In one aspect, the application provides a information recommendation method based on object association data, which comprises:

[0007] When the first terminal and the second terminal are located in the same target area range, the first terminal sends notification information of the object entering the target mode to the second terminal in response to the target mode selection instruction triggered by the object based on the mode selection interface;

[0008] The first terminal collects target feature data related to the target mode in response to the object feature data collection instruction triggered by the object based on the information collection interface, and sends the target feature data to the server;

[0009] The second terminal receives the notification information, and sends a play request for obtaining the play information corresponding to the target mode to the server;

[0010] The server receives the play request, performs object correlation data analysis processing on the target feature data, and obtains a target correlation data analysis result;

[0011] The server determines the play information corresponding to the target correlation data analysis result, and sends the play information to the second terminal;

[0012] The second terminal plays the play information.

[0013] In another aspect, the present application provides an information recommendation system based on object correlation data, which comprises at least a first terminal, a second terminal and a server;

[0014] The first terminal is configured to send notification information of entering the target mode to the second terminal in response to a target mode selection instruction triggered by the object based on a mode selection interface when the first terminal and the second terminal are in the same target area range, and is configured to collect target feature data related to the target mode in response to an object feature data collection instruction triggered by the object based on an information collection interface, and send the target feature data to the server;

[0015] The server is configured to receive a play request for obtaining the play information corresponding to the target mode sent by the second terminal, perform object correlation data analysis processing on the target feature data, and obtain a target correlation data analysis result, and is configured to determine the play information corresponding to the target correlation data analysis result, and send the play information to the second terminal;

[0016] The second terminal is configured to receive the notification information of entering the target mode sent by the first terminal, and is configured to play the play information.

[0017] In another aspect, the present application provides an electronic device comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the information recommendation method based on object correlation data as described above.

[0018] In another aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the information recommendation method based on object correlation data as described above.

[0019] The information recommendation method, system and storage medium based on object correlation data provided by the embodiment of the present application, when the first terminal and the second terminal are located in the same target area range, if the object needs to be recommended to play information, the corresponding mode selection can be performed on the mode selection interface of the first terminal to trigger the corresponding target mode selection instruction, the first terminal responds to the instruction and sends the notification information that the object enters the target mode to the second terminal, at the same time, the object triggers the object feature data collection instruction on the information collection interface of the first terminal, so that the first terminal collects the target feature data related to the target mode and sends the target feature data to the server, the server receives the play request of the second terminal to obtain the play information corresponding to the target mode, analyzes and processes the target feature data to obtain the target correlation data analysis result, determines the play information corresponding to the target correlation data analysis result, and finally sends the play information to the second terminal, and the second terminal plays the play information. The above technical solution of the embodiment of the present application is applied to the smart home scene, and through the linkage of each smart device in the Internet of Things, accurate information recommendation based on object correlation data is realized. At the same time, the target feature data collected by the first terminal is related to the target mode, different target modes will collect different target feature data, so that the collection of target feature data is more representative, thereby improving the accuracy of the determination of the target correlation data analysis result under different target modes, and further improving the accuracy of the play information recommendation under different target modes, so that the recommended play information under different target modes can better meet the current needs of the object in the mode, and the experience is good. In addition, since the recommendation of the play information is related to the target mode, different play information can be recommended for different target modes, and the recommended adaptation range is wider and the flexibility is higher. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0021] Figure 1 is an implementation environment schematic diagram of an information recommendation method based on object correlation data provided by the embodiment of the present application.

[0022] Figure 2 is a flowchart of an information recommendation method based on object correlation data provided by the embodiment of the present application.

[0023] Figure 3is a flowchart of another information recommendation method based on object-related data provided by an embodiment of the present application.

[0024] Figure 4 is a corresponding logic block diagram. Figure 3

[0025] Figure 5 is a flowchart of a first preset information database provided by an embodiment of the present application.

[0026] Figure 6 is a flowchart of another first preset information database provided by an embodiment of the present application.

[0027] Figure 7 is a flowchart of another information recommendation method based on object-related data provided by an embodiment of the present application.

[0028] Figure 8 is a flowchart of another information recommendation method based on object-related data provided by an embodiment of the present application.

[0029] Figure 9 is an optional structure diagram of a blockchain system provided by an embodiment of the present application.

[0030] Figure 10 is an optional diagram of a block structure provided by an embodiment of the present application.

[0031] Figure 11 is a structure diagram of an information recommendation system based on object-related data provided by an embodiment of the present application.

[0032] Figure 12 is a hardware structure block diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION

[0033] With the research and progress of artificial intelligence (AI), AI is researched and applied in multiple fields. AI is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0034] ​Specifically, the scheme provided by the embodiment of the present application relates to a nature language processing (NLP), a computer vision (CV) and a machine learning (ML) technology of artificial intelligence. The NLP is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers in natural language. Natural language processing is a science that integrates linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language used in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question and answer, knowledge graph and the like.

[0035] The CV is a science of how to make a machine "see". Further, it refers to a machine vision that uses a camera and a computer to replace human eyes to identify and measure a target, and further performs image processing, so that the computer processing becomes an image more suitable for human eye observation or transmission to an instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and tries to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping and the like. It also includes common face recognition, fingerprint recognition and other biometric identification technologies.

[0036] The ML is a multi-field cross discipline, which involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. It specially studies how a computer simulates or realizes a learning behavior of human being to obtain new knowledge or skill, and reorganizes an existing knowledge structure to constantly improve the performance of itself. The ML usually includes deep learning, reinforcement learning, transfer learning, inductive learning, teaching learning and the like.

[0037] Specifically, the "server determines the target emotional state information of the object based on the voice data, the facial feature data, the surrounding environment data and the social data" in the embodiment of the present application relates to an emotional analysis technology of voice understanding in the NLP.

[0038] Specifically, the determination of the facial feature data provided by the embodiment of the present application relates to a face recognition technology in the CV and the like.

[0039] Specifically, the information recommendation model based on the server is used to perform information recommendation processing on target motion state information and target emotional state information, and target playing information types are obtained; the information recommendation model is trained based on sample motion state information, corresponding sample emotional state information and corresponding playing information labels, and relates to deep learning technology in ML.

[0040] Cloud technology refers to a kind of hosting technology that unifies a series of resources such as hardware, software and network in a wide area network or a local area network to realize data calculation, storage, processing and sharing.

[0041] Cloud technology is a general term of network technology, information technology, integration technology, management platform technology, application technology and the like applied based on cloud computing business model, can form a resource pool, and is used on demand, flexible and convenient. The background service of a technical network system needs a large amount of calculation and storage resources, such as video websites, picture websites and more portals. With the high development and application of the Internet industry, in the future, every item may have its own identification mark, and needs to be transmitted to the background system for logical processing. Different levels of data will be processed separately, and data of various industries needs strong system support, which can only be realized through cloud computing. Specifically, cloud technology includes technical fields such as security, big data, database, industry application, network, storage, management tool and calculation.

[0042] Specifically, the present application relates to the field of big data technology in cloud technology.

[0043] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0044] It should be noted that the terms "first", "second", and the like in the description and claims of the application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0045] Figure 1 is a schematic diagram of an implementation environment of an information recommendation method based on object-related data provided by an embodiment of the application. As shown in Figure 1 , the implementation environment can at least include a terminal 01 and a server 02, and the terminal 01 and the server 02 can be directly or indirectly connected through wired or wireless communication, which is not limited by the application.

[0046] Specifically, the terminal 01 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart door lock, a smart television, etc., but is not limited thereto.

[0047] Specifically, the server 02 can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms.

[0048] Specifically, the terminal 01 can at least include a first terminal 011, a second terminal 012, a third terminal 013, a fourth terminal 014, and a fifth terminal 015, and each terminal and the server 02 can be directly or indirectly connected through wired or wireless communication.

[0049] Specifically, the first terminal 011 can be a smart wearable device, including but not limited to a smart bracelet, a smart watch, smart glasses, a smart helmet, etc.

[0050] Specifically, the second terminal 012 can be a terminal with voice or music playing function, including but not limited to a smart speaker, a smart television, a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.

[0051] Specifically, the third terminal 013 can be a terminal with a multimedia information (such as a TV series, a movie, a video, etc.) playing function, including but not limited to a smart TV, a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.

[0052] Specifically, the fourth terminal 014 can be a terminal with an electronic book reading function, including but not limited to an electronic book reading terminal, a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.

[0053] Specifically, the fifth terminal 015 can be a terminal with a face or voiceprint recognition function, including but not limited to a smart door lock, etc.

[0054] Specifically, the implementation environment is implemented in a smart home scene, each terminal is a smart device in the smart home scene, and each terminal is bound through Internet of Things technology, respectively.

[0055] Hereinafter, taking the first terminal 011 as a bracelet and the second terminal 012 as a sound box as an example, a feasible binding method provided by the embodiment of the application is introduced.

[0056] 1) The bracelet reads the first identifier of the sound box, the first identifier is an identifier allocated by the manufacturer to the sound box when the sound box is produced, and the first identifier is stored in a near field communication (NFC) chip or a radio frequency identification (RFID) chip of the sound box. The bracelet can read the first identifier through NFC or RFID.

[0057] 2) The bracelet identity document (ID) and the first identifier are sent to the sound box.

[0058] 3) The sound box encrypts or signs the bracelet ID using the first identifier information.

[0059] 4) The sound box returns the encrypted / signature data and the second identifier of the sound box to the bracelet, and the second identifier is the ID of the sound box.

[0060] 5) The bracelet uploads the encrypted / signature data, the bracelet ID, and the device second identifier to the cloud platform to request binding.

[0061] 6) The cloud platform decrypts or verifies the signature according to the second identifier to obtain the bracelet ID.

[0062] 7) The cloud platform judges whether the bracelet ID is legal, if so, the bracelet ID is bound with the sound box, and returns a binding success.

[0063] In the embodiment of the present application, the binding mode between other terminals can refer to the above mode, which will not be repeated here.

[0064] The embodiment of the present application uses the above binding mode, requires a terminal to obtain the first identifier (i.e. the identifier of another terminal) required for binding in close proximity, and uses the first identifier to encrypt the information required for binding on the other terminal side, decrypts and verifies on the cloud platform side, ensuring that illegal objects cannot bind each smart device in the smart home, and improving the security of binding.

[0065] It should be noted that the above binding mode is only an example, and in addition to the above binding mode, other binding modes can also be used, and the embodiment of the present application does not limit this.

[0066] It should be noted that, Figure 1 It is only an example.

[0067] It can be understood that in the specific embodiments of the present application, data related to user information is involved, and when the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0068] Figure 2 is a flowchart of an information recommendation method based on object association data provided by the embodiment of the present application. The method can be used in the implementation environment in Figure 1 The present specification provides method operation steps as described in the embodiments or flowcharts, but more or fewer operation steps can be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one of the many step execution orders, and does not represent the only execution order. When the system or server product is executed in practice, it can be executed in sequence or in parallel (such as parallel processor or multi-threaded processing environment) according to the method order shown in the embodiments or drawings. Specifically as shown in Figure 2 The method can include:

[0069] S101. When the first terminal and the second terminal are located in the same target area range, the first terminal sends the notification information of the object entering the target mode to the second terminal in response to the target mode selection instruction triggered by the object based on the mode selection interface.

[0070] S103. The first terminal collects target feature data related to the target mode in response to the object feature data collection instruction triggered by the object based on the information collection interface, and sends the target feature data to the server.

[0071] S105. The second terminal receives the notification information, and sends a play request for obtaining the play information corresponding to the target mode to the server.

[0072] S107. The server receives the play request, performs object correlation data analysis processing on the target feature data, and obtains a target correlation data analysis result.

[0073] S109. The server determines the play information corresponding to the target correlation data analysis result, and sends the play information to the second terminal.

[0074] In the embodiment of the application, the play information includes but is not limited to voice information and / or music, etc.

[0075] S1011. The second terminal plays the play information.

[0076] The application scenario in the embodiment of the application is a smart home scenario, and accordingly, the target area range can be a house corresponding to the object, i.e., an internal area of a house in which the object lives. When the first terminal and the second terminal are both located in the house corresponding to the object, if the object expects the linkage of each smart device in the house, so as to recommend the play information in the corresponding mode, the object can make a corresponding mode selection on the mode selection interface of the first terminal to trigger a corresponding target mode selection instruction, the first terminal responds to the instruction and sends notification information of the object entering the target mode to the second terminal, at the same time, the object triggers an object feature data collection instruction on the information collection interface of the first terminal, so as to make the first terminal collect target feature data related to the target mode, and send the target feature data to the server. The server receives the play request for obtaining the play information corresponding to the target mode sent by the second terminal, performs object correlation data analysis processing on the target feature data, obtains a target correlation data analysis result, determines the play information corresponding to the target correlation data analysis result, and finally sends the play information to the second terminal, which plays the play information.

[0077] The above technical solution of the embodiment of the application is applied to a smart home scenario, and through the linkage of each smart device in the Internet of Things, the precise information recommendation based on object correlation data is realized. Meanwhile, the target feature data collected by the first terminal is related to the target mode, different target modes will collect different target feature data, so that the collection of the target feature data is more representative, thereby improving the accuracy of the determination of the target correlation data analysis result in different target modes, and further improving the accuracy of the play information recommendation in different target modes, so that the recommended play information in different target modes can better meet the current demand of the object in the mode, and the object has a good experience. In addition, the recommendation of the play information is related to the target mode, i.e., different play information can be recommended for different target modes, and the recommended adaptation range is wider and the flexibility is higher.

[0078] Figure 3 Fig. 2 shows a flowchart of another information recommendation method based on object-related data provided by an embodiment of the present application, Figure 4 Fig. 3 shows a corresponding logic diagram. In the information recommendation method based on object-related data, the target mode is a personalized recommendation mode, and as shown in Figure 3 and Figure 4 S101 can include:

[0079] S101011. When the first terminal and the second terminal are located in the same target area range, the first terminal sends notification information that the object enters the personalized recommendation mode to the second terminal in response to a personalized recommendation mode selection instruction triggered by the object based on the mode selection interface.

[0080] In actual application, as shown in Figure 1 the first terminal can be a smart wearable device, including but not limited to a smart bracelet, a smart watch, smart glasses, a smart helmet, etc. The second terminal can be a terminal with voice or music playing function, including but not limited to a smart speaker, a smart TV, a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.

[0081] Hereinafter, taking the first terminal as a bracelet, the second terminal as a speaker, and the playing information as music as an example, S101011 is described:

[0082] When the bracelet and the speaker are located in the same target area range (i.e. the bracelet and the speaker are both in the object's corresponding home), if the object wants the speaker to make personalized music recommendation, the object wears the bracelet, enters the mode selection interface of the bracelet, selects the personalized recommendation mode on the mode selection interface of the bracelet, or inputs voice information such as "enter the personalized recommendation mode", to trigger a personalized recommendation mode selection instruction, and then the bracelet sends notification information that the object enters the personalized recommendation mode to the speaker in response to the personalized recommendation mode selection instruction, to inform the speaker that the object enters the personalized recommendation mode.

[0083] Correspondingly, as shown in Figure 3 and Figure 4 S103 can include:

[0084] S103011. The first terminal acquires first physiological data, voice data, facial feature data, surrounding environment data and social data in response to an object feature data acquisition instruction triggered by the object in the form of voice based on the information collection interface; the voice data is voice information carried in the voice triggering the object feature data acquisition instruction, the facial feature data is the facial information of the person in the preset photo, the surrounding environment data is the environmental information in the preset photo, and the preset photo is a photo taken within a first preset time acquired from the object's album.

[0085] S103013. The first terminal sends the target feature data to the server.

[0086] S103015. The first terminal sends the target feature data to the server.

[0087] The following takes the first terminal as a bracelet, the second terminal as a sound box, and the playing information as music as an example to illustrate S103011-S103015:

[0088] The object wears the bracelet and enters the information collection interface of the bracelet, and inputs voice information such as "please collect object feature data" in the information collection interface to trigger the object feature data collection instruction. The bracelet responds to the object feature data collection instruction, acquires the first physiological data, voice data, facial feature data, and social data, and sends the first physiological data, voice data, facial feature data, surrounding environment data, and social data as target feature data related to the personalized recommendation mode to the server. The voice data can also be voice information such as "enter the personalized recommendation mode" input by the object in the mode selection interface of the bracelet in S101011.

[0089] Specifically, the physiological data includes but is not limited to heart rate data, respiration data, blood oxygen data, body temperature data, etc.

[0090] Specifically, the process of acquiring the facial feature data can be as follows: the bracelet acquires a photo taken within a first preset time (such as the current day) from an object album (such as a cloud album), takes the photo as a preset photo, takes the facial information of a person (i.e., the object) in the preset photo as the facial feature data, and the facial feature data includes but is not limited to eye corner shape, lip corner shape, lip shape, nose shape, eyebrow shape, etc.

[0091] Specifically, the process of acquiring the surrounding environment data can be as follows: the bracelet acquires environment information in the preset photo from the preset photo, takes the environment information as the surrounding environment information, and the surrounding environment information includes but is not limited to surrounding environment location, surrounding environment color brightness, surrounding environment color quantity, etc.

[0092] In the embodiment of the present application, on the one hand, the target feature data collected by the first terminal is related to the target mode, and different target modes collect different target feature data, so that the collection of target feature data is more representative, thereby improving the accuracy of the determination of the target correlation data analysis result under different target modes, and further improving the accuracy of the recommended playing information under different target modes, so that the recommended playing information under different modes can better meet the needs of the object in the mode, and the object has a good experience. On the other hand, since the physiological data can represent the motion state information of the object, collecting the physiological data can be used as a basis for subsequent determination of the motion state, thereby improving the accuracy of the determination of the motion state information. On the other hand, since the voice data, facial feature data, surrounding environment data of the environment where the object is located, and social data of the object can represent the emotional state information of the object, collecting the voice data, facial feature data, surrounding environment data, and social data can be used as a basis for subsequent determination of the emotional state, thereby improving the accuracy of the determination of the emotional state information.

[0093] Correspondingly, continuing as shown in Figure 3 S105 can include:

[0094] S105011. The second terminal receives the notification information and sends a playing request for obtaining the playing information corresponding to the personalized recommendation mode to the server.

[0095] Taking the second terminal as an example, when the sound box receives the notification information, it sends a playing request for obtaining the playing information corresponding to the personalized recommendation mode to the server.

[0096] Correspondingly, continuing as shown in Figure 3 and Figure 4 S107 can include:

[0097] S107011. The server receives the playing request, matches the first physiological data with the preset motion threshold range, and when the duration of the first physiological data within the preset motion threshold range is greater than the third preset time, the motion state corresponding to the preset motion threshold range is taken as the target motion state information.

[0098] In actual application, the motion state includes but is not limited to motion state, calm state, meditation state, etc., and the mapping relationship between different preset motion threshold ranges and motion state information can be pre-set, such as the preset motion threshold range A1-B1 corresponding to the intense motion state, the preset motion threshold range A2-B2 corresponding to the slow motion state, the preset motion threshold range A3-B3 corresponding to the calm state, the preset motion threshold range A4-B4 corresponding to the meditation state, etc.

[0099] Taking the first physiological data as the heart rate data as an example, after the server receives the playing request, the heart rate data can be compared with the preset running threshold range to determine which range the heart rate data is in. In order to improve the accuracy of the motion state judgment and avoid the error caused by the instantaneous heart rate data to the motion state information judgment, after it is determined that the heart rate data is in a certain preset motion threshold range, the motion state corresponding to the preset motion threshold range is not directly taken as the target motion state information, but the duration that the heart rate data is in the preset motion threshold range is further determined. If the duration is greater than the third preset time, it is determined that the motion state corresponding to the preset motion threshold range is the target motion state information, thereby improving the accuracy of the target motion state information determination and further improving the accuracy of the subsequent playing information recommendation.

[0100] In some embodiments, S107013 can include: determining, by the server, target emotion state information of the object based on voice data, facial feature data, surrounding environment data, and social data.

[0101] In some embodiments, S107013 can include: determining, by the server, target emotion state information of the object based on voice data, facial feature data, surrounding environment data, and social data.

[0102] In this embodiment, sample data (including sample voice data, sample facial feature data, sample surrounding environment data, and sample social data) can be collected in advance, and corresponding emotion labels are labeled on the sample data. Then, the neural network model is trained through the sample data. In the model training process, the parameters of the neural network model are constantly adjusted until the emotion labels output by the neural network model match the emotion labels labeled in the sample data, thereby obtaining the emotion recognition model.

[0103] In other embodiments, S107013 can further include:

[0104] The server extracts features from the voice data to obtain at least one audio feature corresponding to the voice data, and determines voice emotion state information corresponding to the voice information based on the at least one audio feature. The audio features include but are not limited to pitch features, loudness features, speech rate features, timbre features, etc.

[0105] The server extracts features from the facial feature data to obtain facial emotion state information corresponding to the facial feature data.

[0106] The server analyzes the environment features of the surrounding environment data to obtain environment emotion state information corresponding to the surrounding environment data.

[0107] The server extracts emotional text from social data to describe emotions, and determines the corresponding social emotional state information based on the emotional text. This emotional text includes, but is not limited to: happy, sad, upset, angry, crying, laughing, feeling both happy and sad, and joyful.

[0108] The server fuses voice emotion state information, facial emotion state information, environmental emotion state information, and social emotion state information to obtain the target emotion state information.

[0109] In practical applications, the target's emotional state information includes, but is not limited to: excitement, happiness, sadness, grief, frustration, anger, and pleasure.

[0110] This embodiment determines the target emotional state information of an object through the combined effects of voice data, facial feature data, surrounding environment data, and social data. In determining this emotional state information, not only are the object's facial expressions considered, but also the surrounding environment data in the captured photos can reflect the object's emotions to some extent. Therefore, combining the surrounding environment data to further determine the emotional state information can improve the accuracy of the target emotional state information determination, thereby improving the accuracy of subsequent personalized playback information recommendations. Furthermore, since this embodiment of the invention is set in a smart home scenario, the emotions of family members can directly or indirectly affect the object's emotions. Therefore, social data from family groups within instant messaging software can also be combined to further determine the emotional state, thereby further improving the accuracy of the target emotional state information determination and, consequently, the accuracy of subsequent personalized playback information recommendations.

[0111] S107015. The server uses the target's motion state information and the target's emotional state information as the results of target association data analysis.

[0112] After the server determines the target's motion state information and emotional state information, the target's motion state information and emotional state information can be used as the target association data analysis result (which is equivalent to a profile of the object) for subsequent use.

[0113] Accordingly, continue as Figure 3 As shown, S109 may include:

[0114] S109011. The server performs information recommendation processing on the target motion state information and target emotional state information based on the information recommendation model to obtain the target playback information type; the information recommendation model is trained based on the sample motion state information, the corresponding sample emotional state information, and the corresponding playback information labels.

[0115] In the embodiment of the present application, taking music as an example of the playing information, an information recommendation model can be established in advance, and the establishment process of the information recommendation model can be as follows:

[0116] Sample data of the object (including sample motion state information in a certain historical time period of the object and sample emotional state information in the historical time period) is obtained in advance, and the sample data is labeled with corresponding playing information tags. The process of labeling the sample data with corresponding playing information tags can be as follows: assuming that the sample motion state information is an intense motion state and the sample emotional state information is a happy state, the playing information tag can be a happy type of music with a fast rhythm; assuming that the sample motion state information is a slow motion state and the sample emotional state information is a sad state, the playing information tag can be a healing type of music with a moderate rhythm; assuming that the sample motion state information is a calm state and the sample emotional state information is a pleasant state, the playing information tag can be a relaxed type of music with a slow rhythm, and so on.

[0117] After the labeling is completed, the neural network model can be trained through the sample motion state information and the sample emotional state information. In the model training process, the parameters of the neural network model are continuously adjusted until the playing information tag output by the neural network model matches the playing information tag labeled in the sample data, so as to obtain the information recommendation model.

[0118] After the information recommendation model is established, the target motion state information and the target emotional state information can be input into the information recommendation model, so as to obtain a target playing information type corresponding to the target motion state information and the target emotional state information. Assuming that the target motion state information is an intense motion state and the target emotional state information is a happy state, the target playing information type is a happy type of music with a fast rhythm; assuming that the target sample motion state information is a slow motion state and the target emotional state information is a sad state, the target playing information type is a healing type of music with a moderate rhythm; assuming that the target motion state information is a calm state and the target emotional state information is a pleasant state, the target playing information type is a relaxed type of music with a slow rhythm, and so on.

[0119] The information recommendation model in the embodiment of the present application is trained by the sample motion state information and the corresponding sample emotional state information, that is, the information recommendation model is a result of the combination of the motion state information and the emotional state information, that is, the determination of the target playing information type is not only related to the motion state information of the object, but also related to the emotional state information of the object, and the accuracy of the determination of the target playing information type is high, thereby improving the accuracy of subsequent playing information recommendation, making the recommended playing information more meet the current needs of the object, and further improving the experience of the object.

[0120] S109013. The server obtains a candidate playing information set corresponding to the target playing information type from the first preset information library.

[0121] After determining the target playing information type, the server can acquire a candidate playing information set corresponding to the target playing information type from the first preset information library for subsequent use.

[0122] In some embodiments, the method further includes the step of acquiring the first preset information library, as shown in Figure 5 The acquisition of the first preset information library can include:

[0123] S201. The third terminal plays the preset multimedia information in response to the subject triggering a playing instruction of the preset multimedia information based on the multimedia information playing interface within the fourth preset time, and sends the identification information of the preset multimedia information to the server.

[0124] S203. The server acquires the theme music corresponding to the identification information.

[0125] S205. The server stores the theme music as playing information to obtain the first preset information library.

[0126] In this embodiment, as shown in Figure 1 The third terminal can be a terminal with a multimedia information (such as a TV series, a movie, a video, etc.) playing function, including but not limited to a smart TV, a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.

[0127] Taking the third terminal as a smart TV and the multimedia playing information as a movie as an example, S201-S205 are described as follows:

[0128] Within the fourth preset time, if the subject wants to watch a movie, the subject can select the movie to be watched on the multimedia information playing interface of the smart TV to trigger the playing instruction of the preset multimedia information. The smart TV responds to the instruction, plays the movie, and sends the identification information of the movie to the server. The server searches for the theme music corresponding to the identification information and stores the theme music to obtain the first preset information library. That is, the first preset information library can be composed of theme songs of the movies watched by the subject within the fourth preset time.

[0129] In the embodiment of the application, after determining the target playing information type, the candidate playing information set corresponding to the target playing information type (the candidate playing information set is composed of movie theme songs) can be acquired from the first preset information library. Assuming that the target playing information type is a soothing and relaxing type of music, several theme songs of the type of "soothing and relaxing type of music" can be screened from the first preset information library, and the several theme songs are taken as the candidate playing information set.

[0130] Since the movies watched by the object within a time period can reflect the emotional state, the mental journey and the interested things of the object within a time period to some extent, the theme song of the movie watched by the object within a time period is taken as the basis for determining the candidate playing information set, which is a preliminary screening of the playing information, which determines the efficiency of the subsequent sorting stage and the pros and cons of the final recommendation result to some extent, thereby improving the accuracy of the subsequent playing information determination, so that the recommended playing information can meet the needs of the object, and the object has a good experience.

[0131] In some other embodiments, as shown in Figure 6 The first preset information library can also include:

[0132] S301. The fourth terminal displays the preset book in response to the reading instruction of the preset book triggered by the object on the electronic book display interface within the fifth preset time, and sends the book type of the preset book to the server.

[0133] S303. The server acquires music corresponding to the book type.

[0134] S305. The server stores the music corresponding to the book type as playing information to obtain the first preset information library.

[0135] In this embodiment, as shown in Figure 1 The fourth terminal can be a terminal with electronic book reading function, including but not limited to electronic book reading terminal, smart phone, tablet computer, notebook computer, desktop computer, etc.

[0136] Taking the fourth terminal as an example, the electronic book reading terminal, S301-S305 are described as follows:

[0137] Within the fifth preset time, if the object wants to watch the electronic book, the electronic book reading terminal can be selected on the electronic book display interface of the electronic book reading terminal to trigger the reading instruction of the preset book, and the electronic book reading terminal responds to the instruction to display the preset electronic book and sends the book type of the preset electronic book to the server. The server searches for music corresponding to the book type and stores the music corresponding to the book type to obtain the first preset information library. Assuming that the book type is a traditional historical book, the corresponding music can be a nostalgic Chinese music, assuming that the book type is a love type emotional book, the corresponding music can be a sweet type music, and assuming that the book type is a book of entrepreneurship theme, the corresponding music can be a music of the type of inspiration, etc.

[0138] In the embodiment of the present application, after the target playing information type is determined, the candidate playing information set corresponding to the target playing information type can be obtained from the first preset information base (the candidate playing information set is a set of music corresponding to the book type of the electronic book read by the object). Assuming that the target playing information type is music of the inspirational type, music corresponding to several book types of the type of "music of the inspirational type" can be filtered out from the first preset information base, and the music corresponding to the several book types is taken as the candidate playing information set.

[0139] Since the books watched by the object within a certain period of time can reflect the emotional state, the course of mind and the interested things of the object within a certain period of time to some extent, the music corresponding to the book type watched by the object within a certain period of time is taken as the basis for determining the candidate playing information set, the candidate playing information set is equivalent to a preliminary screening of the playing information, which determines the efficiency of the subsequent sorting stage and the pros and cons of the final recommendation result to some extent, thereby improving the accuracy of the subsequent playing information determination, so that the recommended playing information can meet the needs of the object, and the object has a good experience.

[0140] S109015. The server obtains the playing score of each candidate playing information in the candidate playing information set.

[0141] S109017. The server sorts each candidate playing information in reverse order according to the corresponding playing score to obtain a playing information sequence.

[0142] S109019. The server takes the first preset number of playing information in the playing information sequence as the first target playing information.

[0143] S1090111. The first target playing information is sent to the second terminal.

[0144] In actual application, since each candidate playing information will have a corresponding score on the corresponding network platform (the score can be the score after the ordinary object finishes listening to the playing information, or the score by the professional person), the server can obtain the playing score of each candidate playing information from the corresponding platform, sort each candidate playing information in reverse order according to the corresponding playing score to obtain a playing information sequence, and finally take the playing information with a higher score (i.e., the playing information at the front of the sequence) as the first target playing information and recommend it to the second terminal.

[0145] Correspondingly, continue as shown in Figure 3 and Figure 4 S1011 can include:

[0146] S1011011. The second terminal plays the first target playing information.

[0147] In this embodiment of the invention, after obtaining the candidate playback information set, the set is not directly recommended to the second terminal. Instead, the candidate playback information is sorted according to its rating, and only the top-ranked (i.e., high-rated) pieces are recommended as the first target playback information to the second terminal, which then plays the first target playback information. This not only allows for accurate music recommendations but also ensures that high-quality music is recommended, improving the overall quality of music recommendations and thus enhancing the user experience.

[0148] As described in S101011-S1011011, the embodiments of the present invention achieve personalized and accurate information recommendation based on object association data (movement state + emotional state) in a smart home scenario through the linkage between the first terminal, the second terminal, and the server.

[0149] Figure 7 The diagram shown is a flowchart illustrating another information recommendation method based on object-related data provided in an embodiment of the present invention. In this information recommendation method based on object-related data, the target mode is the sleep mode, then as follows... Figure 7 As shown, S101 may include:

[0150] S101031. When the first terminal and the second terminal are located within the same target area, the first terminal responds to the sleep mode selection instruction triggered by the object based on the mode selection interface and sends a notification message to the second terminal that the object has entered sleep mode.

[0151] In practical applications, such as Figure 1 As shown, the first terminal can be a smart wearable device, including but not limited to smart bracelets, smartwatches, smart glasses, and smart helmets. The second terminal can be a terminal with voice or music playback functions, including but not limited to smart speakers, smart TVs, smartphones, tablets, laptops, and desktop computers.

[0152] The following explanation uses a wristband as the first terminal, a speaker as the second terminal, and music playback information as an example:

[0153] When the wristband and speaker are located in the same target area (for example, both the wristband and speaker are in the user's home), if the user wants the speaker to recommend sleep music, the user can wear the wristband and enter the wristband's mode selection interface. On the wristband's mode selection interface, the user can select sleep mode to trigger the sleep mode selection command. In response to the sleep mode selection command, the wristband will send a notification message to the speaker informing the speaker that the user has entered sleep mode.

[0154] Accordingly, continue as Figure 7S103 can include:

[0155] S103031. The first terminal collects second physiological data related to the sleep mode in response to the object feature data collection instruction triggered by the object based on the information collection interface, and takes the second physiological data as target feature data related to the sleep mode.

[0156] S103033. The first terminal sends the target feature data to the server.

[0157] Hereinafter, taking the first terminal as a bracelet, the second terminal as a sound box, and the playing information as music as an example, S103031-S103033 are described:

[0158] The object can enter the information collection interface of the bracelet, and voice input "please collect physiological data" voice information or manually select the collection button in the information collection interface to trigger the object feature data collection instruction. The bracelet responds to the object feature data collection instruction to obtain second physiological data related to the sleep mode.

[0159] Specifically, the second physiological data includes but is not limited to heart rate data, respiration data, blood oxygen data, body temperature data, etc.

[0160] Accordingly, continue as Figure 7 S105 can include:

[0161] S105031. The second terminal receives the notification information and sends a playing request for obtaining the playing information corresponding to the sleep mode to the server.

[0162] Taking the second terminal as a sound box as an example, when the sound box receives the notification information, it sends a playing request for obtaining the playing information corresponding to the sleep mode to the server.

[0163] Accordingly, continue as Figure 7 S107 can include:

[0164] S107031. The server receives the playing request, matches the second physiological data with the preset sleep threshold range, and when the duration of the second physiological data in the preset sleep threshold range is greater than the sixth preset time, takes the sleep state corresponding to the preset sleep threshold range as the target sleep state, and takes the target sleep state as the target correlation data analysis result.

[0165] Taking the heart rate data as an example, after the server receives the playing request, the heart rate data can be compared with the preset sleep threshold range to determine which range the heart rate data is in. In order to improve the accuracy of the target sleep state judgment and avoid the error caused by the instantaneous heart rate data to the target sleep state judgment, after it is determined that the heart rate data is in a certain preset sleep threshold range, the sleep state corresponding to the preset sleep threshold range is not directly taken as the target sleep state, but the duration that the heart rate data is in the preset sleep threshold range is further determined. If the duration is greater than the sixth preset time, the sleep state corresponding to the preset sleep threshold range is determined as the target sleep state, thereby improving the accuracy of sleep state determination and further improving the accuracy of subsequent playing information recommendation.

[0166] Correspondingly, S109 can include: Figure 7

[0167] S109031. The server obtains third target playing information corresponding to the target sleep state from a third preset information library, and sends the third target playing information to the second terminal. The third preset information library stores a mapping relationship between sleep states and playing information.

[0168] S109033. The server sends the third playing information to the second terminal.

[0169] The above step S1011 can include:

[0170] S1011031. The second terminal plays the third playing information.

[0171] In actual application, the sleep state can include a sleep-in state, a light sleep state and a deep sleep state. A mapping relationship between each sleep state and playing information can be established in advance, and the mapping relationship is stored in the third preset information library.

[0172] In some embodiments, the mapping relationship can be that a plurality of sleep states correspond to the same music: the sleep-in mode corresponds to elegant and quiet music, the light sleep state adjusts the sound of the elegant and quiet music to a preset volume threshold within a preset time (i.e. the sound of the elegant and quiet music gradually weakens), and the deep sleep state stops playing the elegant and quiet music.

[0173] In other embodiments, the mapping relationship can also be that each sleep state corresponds to different music: the sleep-in mode corresponds to light music, the light sleep state corresponds to quiet and hypnotic music, and the deep sleep state does not play music.

[0174] ​In the embodiment of the present application, by setting different sleep states corresponding to different playing information, appropriate sleep music can be accurately pushed to the object according to the sleep state, so as to improve the sleep quality of the object without disturbing the sleep of the object, and further improve the experience of the object.

[0175] In actual application, if the object sets an alarm, the preset alarm sound set by the object will be played regardless of the sleep state of the object, for example, when the object is having a nightmare, the object suddenly hears the alarm sound of a song in rock music style, which is easy to cause the object to feel disgusted and affect the mood of the object when getting up. In addition, when the alarm sound needs to be changed, the object can only manually set other alarm sounds, thereby reducing the experience of the object.

[0176] Therefore, the embodiment of the present application further provides a method for playing an alarm sound corresponding to a sleep state, specifically comprising:

[0177] The mapping relationship between the sleep state and the alarm sound is set in advance, for example, the deep sleep state corresponds to soothing music with gradually increasing sound, the light sleep state corresponds to light music, and the sleep state corresponds to quiet music.

[0178] When the server determines that the current time is the preset alarm time, the alarm sound corresponding to the current sleep state of the object is obtained, and the alarm sound is sent to the second terminal, so that the second terminal plays the alarm sound.

[0179] In the embodiment of the present application, different music styles of songs can be selected as alarm sounds to wake up the object according to the sleep level, so as to cheer up the mood of the object and effectively improve the experience of the object.

[0180] As described in S101031-S1011031, the embodiment of the present application realizes accurate information recommendation of the sleep mode based on the object associated data (sleep state) through the linkage among the first terminal, the second terminal and the server in the smart home scene.

[0181] Figure 8 As shown in the flowchart of another information recommendation method based on object associated data provided by the embodiment of the present application, when the first terminal and the second terminal are not located in the same target area range, the method further comprises: Figure 8 As shown, the method further comprises:

[0182] S0011. The fifth terminal collects the identity information of the object in response to the identity collection instruction triggered by the object based on the identity verification interface, and sends the identity information of the object to the server.

[0183] S0013. The server matches the identity information of the object with the preset identity information, and when the matching is successful, obtains the second target playing information corresponding to the successful identity matching from the second preset information database.

[0184] S0015. The server sends the second target playing information to the second terminal.

[0185] S0017. The second terminal plays the second target playing information.

[0186] In the embodiment of the present application, as shown in the figure, the fifth terminal can be a terminal with face or voiceprint recognition function, including but not limited to smart door lock, etc. Figure 1

[0187] Taking the first terminal as a bracelet, the fifth terminal as a smart door lock, and the second terminal as a sound box as an example, S0011-S0017 are described:

[0188] The object can pre-establish a mapping relationship between identity matching success and the second target playing information, and store the mapping relationship in the second preset information base, for example, identity verification corresponds to cheerful type welcome music, or identity verification corresponds to the welcome voice defined by the object (such as welcome rock prince home).

[0189] When the bracelet and the sound box are not in the same target area range, it means that the object wearing the bracelet is not at home, while the sound box is at home. When the object needs to enter the home, the fingerprint collection, voice collection or face recognition function can be selected in the identity verification interface of the smart door lock to trigger the identity collection instruction. The smart door lock responds to the instruction, collects the fingerprint, voiceprint or face feature of the object from the voice, and matches the collected object identity information with the saved preset identity information. If the matching is successful, it means that the identity recognition is passed (i.e. it is detected that the object wearing the bracelet returns home, at this time it can be considered that the bracelet and the sound box are in the same target area range). After the identity verification is passed, the server obtains the second target playing information corresponding to the identity matching success from the second preset information base, and sends the second target playing information to the sound box. The sound box plays the second target playing information.

[0190] In the embodiment of the present application, by recognizing the identity of the object when the object enters the door (i.e. when it is detected that the object returns home), and playing cheerful type welcome music or welcome voice defined by the object after identity verification is successful, the interest of using the smart door lock is improved, the function of the smart door lock is enriched, the use value of the smart door lock is improved, the interaction between the smart door lock and the person is realized, in addition, the mood of the object is improved, and the experience of the object is improved.

[0191] As described in S0011-S0017, the embodiment of the present application realizes the precise information recommendation of the door lock wake-up mode based on the object associated data (detecting that the object returns home) in the smart home scene through the linkage among the fifth terminal, the second terminal and the server.​

[0192] In one possible embodiment, at least one of the target feature data in S103, the target correlation data analysis result in S107, the playing information in S109, the first physiological data in S103011, the voice data, the facial feature data, the surrounding environment data and the social data, the target motion state information in S107011, the target emotion state information in S107013, the second physiological data in S103031, the first target playing information in S109019, the second target playing information in S0013, and the third target playing information in S109031 can be stored in the blockchain system. Referring to Figure 9 , Figure 9 Fig. 1 shows a possible structure of the blockchain system according to an embodiment of the present application. A plurality of nodes form a peer-to-peer (P2P) network, and the P2P protocol is an application layer protocol running on the transmission control protocol (TCP) protocol. In the blockchain system, any machine such as a server or a terminal can join as a node, and the node includes a hardware layer, an intermediate layer, an operating system layer, and an application layer.

[0193] Referring to Fig. 2, the functions of the nodes in the blockchain system are shown, and the functions involved include: Figure 9

[0194] 1) Routing, a basic function of the node, used to support communication between nodes.

[0195] In addition to the routing function, the node can also have the following functions:

[0196] 2) Application, used to deploy in the blockchain, to implement specific business according to actual business needs, to record the data related to the implementation function to form record data, to carry a digital signature in the record data to indicate the source of the task data, to send the record data to other nodes in the blockchain system, and to add the record data to the temporary block when the other nodes successfully verify the source and integrity of the record data.

[0197] 3) Blockchain, including a series of blocks (Block) connected in the order of time of generation. Once a new block is added to the blockchain, it will not be removed. The block records the record data submitted by the nodes in the blockchain system.

[0198] Referring to Fig. 3, the functions of the nodes in the blockchain system are shown, and the functions involved include: Figure 10 , Figure 10 ​An optional schematic diagram of a block structure provided by the embodiment of the present application is shown in the figure, each block includes a hash value of a transaction record stored in the block (hash value of the block) and a hash value of a previous block, and each block is connected by a hash value to form a block chain.

[0199] The information recommendation method based on object correlation data provided by the embodiment of the present application is applied to the smart home scene, and the personalized and accurate information recommendation based on object correlation data (motion state + emotional state) is realized through the linkage among the first terminal, the second terminal and the server, the linkage among the first terminal, the second terminal and the server is used to realize the sleep mode accurate information recommendation based on object correlation data (sleep state), and the linkage among the fifth terminal, the second terminal and the server is used to realize the accurate information recommendation of the door lock wake-up mode based on object correlation data (detection of the object returning home). The beneficial effects of the information recommendation method based on object correlation data can be as follows:

[0200] 1) The target feature data collected by the first terminal is related to the target mode, different target modes will collect different target feature data, so that the collection of target feature data is more representative, thereby improving the accuracy of the target correlation data analysis result determination in different target modes, and further improving the accuracy of the playing information recommendation in different target modes, so that the playing information recommended in different target modes can better meet the current needs of the object in the mode, and the object has a good experience.

[0201] 2) In order to improve the accuracy of motion state judgment and avoid the error caused by instantaneous heart rate data to the motion state information judgment, after it is determined that the heart rate data is in a certain preset motion threshold range, the motion state corresponding to the preset motion threshold range is not directly used as the target motion state information, but the duration that the heart rate data is in the preset motion threshold range is further judged, if the duration is greater than the third preset time, the motion state corresponding to the preset motion threshold range is determined as the target motion state information, thereby improving the accuracy of the target motion state information determination, and further improving the accuracy of the subsequent playing information recommendation.

[0202] 3) The target emotional state information of the object is determined by the joint action of voice data, facial feature data, surrounding environment data and social data. In the process of determining the emotional state information, not only the facial expression of the object is considered, but also the surrounding environment data in the photographed photo can reflect the emotion of the object to a certain extent. Therefore, the emotional state information is further judged in combination with the surrounding environment data, which can further improve the accuracy of the determination of the target emotional state information, and further improve the accuracy of the subsequent personalized playing information recommendation. In addition, since the environment of the embodiment of the present application is a smart home scene, the emotions of family members will directly or indirectly affect the emotions of the object, so the social data in the family group in the instant messaging software can be combined to further judge the emotional state, thereby further improving the accuracy of the determination of the target emotional state information, and further improving the accuracy of the subsequent personalized playing information recommendation.

[0203] 4) The information recommendation model is trained by the sample motion state information and the corresponding sample emotional state information, that is, the information recommendation model is the result of the combination of the motion state information and the emotional state information. That is, the determination of the target playing information type is not only related to the motion state information of the object, but also related to the emotional state information of the object. The accuracy of the determination of the target playing information type is high, thereby improving the accuracy of the subsequent playing information recommendation, so that the recommended playing information can better meet the current needs of the object, and the experience of the object is improved.

[0204] 5) Since the movies watched by the object in a certain time period or the books watched by the object in a certain time period can reflect the emotional state, the course of the mind and the interested things of the object to a certain extent, the theme song of the movie watched by the object in a certain time period or the music corresponding to the type of the book watched by the object in a certain time period is used as the basis for determining the candidate playing information set. The candidate playing information set is a preliminary screening of the playing information, which determines the efficiency of the subsequent sorting stage and the pros and cons of the final recommendation result to a certain extent, thereby improving the accuracy of the subsequent playing information determination, so that the recommended playing information can meet the needs of the object, and the experience of the object is good.

[0205] 6) After obtaining the candidate playing information set, the candidate playing information set is not directly recommended to the second terminal, but each candidate playing information is sorted according to the score of the candidate playing information. The first target playing information in the front (i.e. with high score) is recommended to the second terminal, and the first target playing information is played by the second terminal. In this way, not only can the object be accurately recommended to the object, but also the music with high quality can be recommended to the object, thereby improving the quality of the music recommendation, and further improving the experience of the object.

[0206] 7) In order to improve the accuracy of the target sleep state judgment, avoid the error caused by the instantaneous heart rate data to the target sleep state judgment, after determining that the heart rate data is in a certain preset sleep threshold range, the sleep state corresponding to the preset sleep threshold range is not directly taken as the target sleep state, but the duration that the heart rate data is in the preset sleep threshold range is continuously judged, if the duration is greater than the sixth preset time, the sleep state corresponding to the preset sleep threshold range is determined as the target sleep state, thereby improving the accuracy of sleep state determination, and further improving the accuracy of subsequent playing information recommendation.

[0207] 8) By setting different playing information corresponding to different sleep states, suitable sleep music can be accurately pushed to the object according to the sleep state, thereby improving the sleep quality of the object on the basis of not disturbing the sleep of the object, and further improving the experience of the object.

[0208] 9) According to the sleep level, a song with different music styles is selected as a ringtone to wake up the object, so as to cheer up the mood of the object and effectively improve the experience of the object.

[0209] 10) By identifying the identity of the object when the object enters the door, and playing cheerful welcome music or welcome voice defined by the object after the identity verification succeeds, the interestingness of the use of the smart door lock is improved, the function of the smart door lock is enriched, the use value of the smart door lock is improved, the interaction between the smart door lock and the person is realized, in addition, the mood of the object is cheered up, and the experience of the object is improved.

[0210] As shown in Figure 11 The embodiment of the application provides an information recommendation system based on object association data, which at least comprises a first terminal, a second terminal and a server, and the system can comprise:

[0211] The first terminal can be used to send notification information of the object entering a target mode to the second terminal in response to a target mode selection instruction triggered by the object based on a mode selection interface when the first terminal and the second terminal are in the same target area range, and can be used to collect target feature data related to the target mode in response to an object feature data collection instruction triggered by the object based on an information collection interface, and send the target feature data to the server.

[0212] The server can be used to receive a playing request for obtaining playing information corresponding to the target mode sent by the second terminal, perform object association data analysis processing on the target feature data to obtain a target association data analysis result, and determine the playing information corresponding to the target association data analysis result and send the playing information to the second terminal.

[0213] The second terminal can be used to receive the notification information of the object entering the target mode sent by the first terminal, and can be used to play the playing information.

[0214] In some embodiments, the target mode is a personalized recommendation mode, the first terminal can further be configured to acquire the first physiological data, voice data, facial feature data, surrounding environment data and social data in response to the object feature data acquisition instruction triggered by the object in the voice form based on the information collection interface; the voice data is voice information carried in the voice triggering the object feature data acquisition instruction, the facial feature data is facial information of a person in a preset photo, the surrounding environment data is environment information in the preset photo, the preset photo is a photo taken within a first preset time acquired from an album of the object; and the first terminal can be configured to take the first physiological data, voice data, facial feature data, surrounding environment data and social data as target feature data related to the personalized recommendation mode.

[0215] Correspondingly, the server can be further configured to receive the play request, match the first physiological data with a preset motion threshold range, take a motion state corresponding to the preset motion threshold range as target motion state information when a duration in which the first physiological data is within the preset motion threshold range is greater than a third preset time, determine target emotion state information of the object based on the voice data, facial feature data, surrounding environment data and social data, and take the target motion state information and the target emotion state information as a target correlation data analysis result.

[0216] Correspondingly, the server can be further configured to perform information recommendation processing on the target motion state information and the target emotion state information based on an information recommendation model to obtain a target play information type; the information recommendation model is obtained by training based on sample motion state information, corresponding sample emotion state information and corresponding play information label; the server can be further configured to acquire a candidate play information set corresponding to the target play information type from a first preset information library, acquire a play score of each candidate play information in the candidate play information set, sort each candidate play information in a reverse order according to the corresponding play score to obtain a play information sequence, take a first preset number of play information in the play information sequence as first target play information, and send the first target play information to the second terminal.

[0217] Correspondingly, the second terminal can be configured to play the first target play information.

[0218] Correspondingly, the system can further include a third terminal. The third terminal can be configured to play preset multimedia information in response to a play instruction of the preset multimedia information triggered by the object based on a multimedia information play interface within a fourth preset time, and send identification information of the preset multimedia information to the server.

[0219] Correspondingly, the server can be configured to acquire theme music corresponding to the identification information, and store the theme music as the playing information to obtain a first preset information base.

[0220] Correspondingly, the system can further include a fourth terminal. The fourth terminal can be configured to, in response to the subject triggering a reading instruction of the preset book based on the electronic book display interface within a fifth preset time, display the preset book, and send a book type of the preset book to the server.

[0221] Correspondingly, the server can be further configured to acquire music corresponding to the book type, and store the music corresponding to the book type as the playing information to obtain the first preset information base.

[0222] In some embodiments, the system can further include a fifth terminal. When the first terminal and the second terminal are not located in the same target area range, the fifth terminal can be configured to, in response to the subject triggering an identity collection instruction based on the identity authentication interface, collect subject identity information, and send the subject identity information to the server.

[0223] Correspondingly, the server can be configured to match the subject identity information with preset identity information, acquire second target playing information corresponding to a successful identity matching from a second preset information base when the matching is successful, and send the second target playing information to the second terminal.

[0224] Correspondingly, the second terminal can be configured to play the second target playing information.

[0225] In some embodiments, the target mode is a sleep mode, and the first terminal can be further configured to, in response to the subject triggering a subject feature data collection instruction based on the information collection interface, collect second physiological data related to the sleep mode.

[0226] Correspondingly, the server can be configured to receive the playing request, match the second physiological data with a preset sleep threshold range, when a duration in which the second physiological data is within the preset sleep threshold range is greater than a sixth preset time, acquire a sleep state corresponding to the preset sleep threshold range as a target sleep state, and acquire a third target playing information corresponding to the target sleep state from a third preset information base, send the third target playing information to the second terminal, and store a mapping relationship between the sleep state and the playing information in the third preset information base.

[0227] Correspondingly, the second terminal can be configured to play the third target playing information.

[0228] It should be noted that the system embodiments provided by the embodiments of the present application are based on the same inventive concept as the above-mentioned method embodiments.

[0229] The embodiments of the present application also provide an electronic device for an information recommendation method based on object association data, which comprises a processor and a memory, and the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the information recommendation method based on object association data provided by the above-mentioned method embodiments.

[0230] The embodiments of the present application also provide a computer readable storage medium, which can be arranged in a terminal to save at least one instruction or at least one program related to the information recommendation method based on object association data in the method embodiments, and the at least one instruction or at least one program is loaded and executed by the processor to implement the information recommendation method based on object association data provided by the above-mentioned method embodiments.

[0231] Optionally, in the embodiments of the present application, the storage medium can be located in at least one of the plurality of network servers of the computer network. Optionally, in the embodiments, the storage medium can include but is not limited to a variety of storage media that can store program codes, such as a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0232] The memory of the embodiments of the present application can be used to store software programs and modules, and the processor executes various function application programs and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by functions, etc.; and the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device or other volatile solid-state memory device. Accordingly, the memory can also include a memory controller to provide access of the processor to the memory.

[0233] The embodiments of the present application also provide a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to make the computer device execute the information recommendation method based on object association data provided by the above-mentioned method embodiments.

[0234] The information recommendation method based on object-related data provided by the embodiments of the present application can be executed in a terminal, a computer terminal, a server or a similar computing device. Taking the case of running on a server, Figure 12 is a hardware structure block diagram of a server of the information recommendation method based on object-related data provided by the embodiments of the present application. As shown in the figure, Figure 12 the server 400 can have a large difference due to different configurations or performances, and can include one or more central processing units (CPU) 410 (the central processing unit 410 can include but not limited to a microprocessor MCU or a programmable logic device FPGA processing device), a memory 430 for storing data, one or more storage media 420 (such as one or more mass storage devices) for storing application programs 423 or data 422. Among them, the memory 430 and the storage medium 420 can be temporary storage or persistent storage. The program stored in the storage medium 420 can include one or more modules, each of which can include a series of instruction operations in the server. Further, the central processing unit 410 can be configured to communicate with the storage medium 420 and execute a series of instruction operations in the storage medium 420 on the server 400. The server 400 can also include one or more power supplies 460, one or more wired or wireless network interfaces 450, one or more input / output interfaces 440, and / or one or more operating systems 421, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM and the like.

[0235] The input / output interface 440 can be used to receive or send data via a network. The above-mentioned specific examples of the network can include a wireless network provided by the communication provider of the server 400. In one example, the input / output interface 440 includes a network adapter (NIC) which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the input / output interface 440 can be a radio frequency (RF) module for communicating with the Internet in a wireless manner.

[0236] Those of ordinary skill in the art can understand that, Figure 12 the structure shown in the figure is only a schematic and does not limit the structure of the above-mentioned electronic device. For example, the server 400 can also include more than Figure 12The more or fewer components shown, or having the same Figure 12 The different configurations shown.

[0237] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0238] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and server embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0239] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0240] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An information recommendation method based on object-related data, characterized by, Applied to a smart home scene, the method comprises: When the first terminal and the second terminal are located in the same target area range, the first terminal sends notification information that the object enters the target mode to the second terminal in response to a target mode selection instruction triggered by the object based on a mode selection interface; the target mode includes a personalized recommendation mode and a sleep mode; and the target area range is a home corresponding to the object; The first terminal collects target feature data related to the target mode in response to an object feature data collection instruction triggered by the object based on an information collection interface, and sends the target feature data to a server; The second terminal receives the notification information and sends a play request for obtaining play information corresponding to the target mode to the server; The server receives the play request, performs object-related data analysis processing on the target feature data, and obtains a target-related data analysis result; when the target mode is the personalized recommendation mode, the target feature data is determined based on first physiological data, voice data, facial feature data, surrounding environment data, and social data of the object, the target-related data analysis result includes target motion state information and target emotional state information, the target motion state information is determined based on a motion state corresponding to a preset motion threshold range when a duration that the first physiological data is within the preset motion threshold range is greater than a third preset time, and the target emotional state information is determined based on the voice data, facial feature data, surrounding environment data, and social data; when the target mode is the sleep mode, the target feature data is second physiological data related to the sleep mode, and the target-related data analysis result is determined based on a sleep state corresponding to a preset sleep threshold range when a duration that the second physiological data is within the preset sleep threshold range is greater than a sixth preset time; the server determines play information corresponding to the target-related data analysis result and sends the play information to the second terminal; and the second terminal plays the play information; When the first terminal and the second terminal are not located in the same target area range, it is determined that the object wearing the first terminal is not at home, and the second terminal is located at home; when it is detected that the object is back home, a fifth terminal collects object identity information in response to an identity collection instruction triggered by the object based on an identity verification interface and sends the object identity information to a server; the server matches the object identity information with preset identity information, and when the matching is successful, it is determined that the first terminal and the second terminal are located in the same target area range, and second target play information corresponding to the identity matching success is obtained from a second preset information base; the server sends the second target play information to the second terminal; the second terminal plays the second target play information; and the fifth terminal includes a smart door lock.

2. The method of claim 1, wherein, The target mode is a personalized recommendation mode, and the first terminal collects target feature data related to the target mode in response to an object feature data collection instruction triggered by the object based on the information collection interface, including: The first terminal acquires first physiological data, voice data, facial feature data, surrounding environment data and social data in response to an object feature data collection instruction triggered by the object in a voice form based on the information collection interface; the voice data is voice information carried in the voice triggering the object feature data collection instruction, the facial feature data is facial information of a person in a preset photo, the surrounding environment data is environment information in the preset photo, and the preset photo is a photo taken within a first preset time acquired from an album of the object; The first terminal takes the first physiological data, the voice data, the facial feature data, the surrounding environment data and the social data as target feature data related to the personalized recommendation mode.

3. The method of claim 2, wherein, The server receives the playing request, performs object correlation data analysis processing on the target feature data, and obtains a target correlation data analysis result, including: The server receives the playing request, matches the first physiological data with a preset motion threshold range, and takes a motion state corresponding to the preset motion threshold range as target motion state information when a duration in which the first physiological data is within the preset motion threshold range is greater than a third preset time; The server determines target emotion state information of the object based on the voice data, the facial feature data, the surrounding environment data and the social data; The server takes the target motion state information and the target emotion state information as the target correlation data analysis result.

4. The method of claim 3, wherein, The server determines playing information corresponding to the target correlation data analysis result, and sends the playing information to a second terminal, including: The server performs information recommendation processing on the target motion state information and the target emotion state information based on an information recommendation model to obtain a target playing information type; the information recommendation model is trained based on sample motion state information, corresponding sample emotion state information and corresponding playing information labels; The server acquires a candidate playing information set corresponding to the target playing information type from a first preset information library; The server acquires a playing score of each candidate playing information in the candidate playing information set; The server performs reverse order sorting on each candidate playing information according to the corresponding playing score to obtain a playing information sequence; The server takes a first preset number of playing information in the playing information sequence as first target playing information; The server sends the first target playing information to the second terminal; Correspondingly, the second terminal plays the playing information, including: The second terminal plays the first target playing information.

5. The method of claim 4, wherein, The method further includes the step of acquiring the first preset information library, and the acquiring of the first preset information library includes: The third terminal plays the preset multimedia information in response to the object triggering a play instruction of the preset multimedia information based on the multimedia information play interface within a fourth preset time, and sends identification information of the preset multimedia information to the server; The server obtains theme music corresponding to the identification information; The server stores the theme music as play information to obtain the first preset information base.

6. The method of claim 4, wherein, The method further includes a step of obtaining the first preset information base, and the step of obtaining the first preset information base includes: The fourth terminal displays the preset book in response to the object triggering a reading instruction of the preset book based on the electronic book display interface within a fifth preset time, and sends a book type of the preset book to the server; The server obtains music corresponding to the book type; The server stores the music corresponding to the book type as play information to obtain the first preset information base.

7. The method of claim 1, wherein, When the target mode is a sleep mode, the first terminal collects target feature data related to the target mode in response to the object triggering an object feature data collection instruction based on an information collection interface, including: The first terminal collects second physiological data related to the sleep mode in response to the object triggering an object feature data collection instruction based on the information collection interface, and takes the second physiological data as the target feature data related to the sleep mode; Correspondingly, the server receives the play request, performs object correlation data analysis processing on the target feature data, and obtains a target correlation data analysis result, including: The server receives the play request, matches the second physiological data with a preset sleep threshold range, takes a sleep state corresponding to the preset sleep threshold range as a target sleep state when a duration in which the second physiological data is within the preset sleep threshold range is greater than a sixth preset time, and takes the target sleep state as the target correlation data analysis result; Correspondingly, the server determines play information corresponding to the target correlation data analysis result, and sends the play information to a second terminal, including: The server obtains third target play information corresponding to the target sleep state from a third preset information base, and sends the third target play information to the second terminal, where the third preset information base stores a mapping relationship between sleep states and play information; Correspondingly, the second terminal plays the play information, including: The second terminal plays the third target play information.

8. An information recommendation system based on object-related data, characterized by In an intelligent home scenario, the system at least includes a first terminal, a second terminal, a fifth terminal, and a server, and the fifth terminal includes an intelligent door lock; The first terminal is configured to, when the first terminal and the second terminal are in a same target area range, send notification information of the object entering the target mode to the second terminal in response to the object triggering a target mode selection instruction based on a mode selection interface. and for collecting target feature data related to the target mode in response to the object triggering an object feature data collection instruction based on the information collection interface, and sending the target feature data to a server; the target mode includes a personalized recommendation mode and a sleep mode; and the target area range is a home corresponding to the object; The server is configured to receive a play request for obtaining play information corresponding to the target mode sent by the second terminal, perform object-associated data analysis processing on the target feature data, and obtain a target-associated data analysis result; and for determining play information corresponding to the target-associated data analysis result and sending the play information to the second terminal; when the target mode is the personalized recommendation mode, the target feature data is determined based on first physiological data, voice data, facial feature data, surrounding environment data, and social data of the object, the target-associated data analysis result includes target motion state information and target emotional state information, the target motion state information is determined based on a motion state corresponding to a preset motion threshold range when a duration in which the first physiological data is within the preset motion threshold range is greater than a third preset time, and the target emotional state information is determined based on the voice data, the facial feature data, the surrounding environment data, and the social data; when the target mode is the sleep mode, the target feature data is second physiological data related to the sleep mode, and the target-associated data analysis result is determined based on a sleep state corresponding to a preset sleep threshold range when a duration in which the second physiological data is within the preset sleep threshold range is greater than a sixth preset time; The second terminal is configured to receive notification information that the object enters the target mode sent by the first terminal; and for playing the play information; When the first terminal and the second terminal are not located in the same target area range, it is determined that the object wearing the first terminal is not at home, and the second terminal is located at home, and when it is detected that the object is back home, the fifth terminal is configured to collect object identity information in response to an identity collection instruction triggered by the object based on an identity verification interface, and send the object identity information to a server; the server is further configured to match the object identity information with preset identity information, and when the matching is successful, determine that the first terminal and the second terminal are located in the same target area range, and obtain second target play information corresponding to the identity matching success from a second preset information base; The second target play information is sent to the second terminal; and the second terminal is further configured to play the second target play information.

9. The information recommendation system according to claim 8, characterized by, The target mode is a personalized recommendation mode, and the first terminal is further configured to: In response to the object feature data collection instruction triggered by the object in a voice form based on the information collection interface, first physiological data, voice data, facial feature data, surrounding environment data, and social data are acquired; the voice data is voice information carried in the voice triggering the object feature data collection instruction, the facial feature data is facial information of a person in a preset photo, the surrounding environment data is environment information in the preset photo, and the preset photo is a photo taken within a first preset time acquired from an album of the object; The first physiological data, the voice data, the facial feature data, the surrounding environment data, and the social data are used as target feature data related to the personalized recommendation mode.

10. The information recommendation system according to claim 9, characterized by, The server is further configured to: receive the playback request, match the first physiological data with a preset exercise threshold range, and when a duration in which the first physiological data is within the preset exercise threshold range is greater than a third preset time, use an exercise state corresponding to the preset exercise threshold range as target exercise state information; determine target emotional state information of the object based on the voice data, the facial feature data, the surrounding environment data, and the social data; use the target exercise state information and the target emotional state information as the target correlation data analysis result.

11. The information recommendation system according to claim 10, characterized by, The server is further configured to: perform information recommendation processing on the target exercise state information and the target emotional state information based on an information recommendation model to obtain a target playback information type; the information recommendation model is trained based on sample exercise state information, corresponding sample emotional state information, and corresponding playback information labels; acquire a candidate playback information set corresponding to the target playback information type from a first preset information base; acquire a playback score of each candidate playback information in the candidate playback information set; perform reverse order sorting on each candidate playback information according to the corresponding playback score to obtain a playback information sequence; use the first preset number of playback information in the playback information sequence as first target playback information; send the first target playback information to the second terminal; The second terminal is further configured to: play the first target playback information.

12. The information recommendation system according to claim 11, wherein, The system further includes a third terminal configured to: in response to an object triggering a playback instruction of preset multimedia information based on a multimedia information playback interface within a fourth preset time, play the preset multimedia information, and send identification information of the preset multimedia information to the server; The server is further configured to: acquire theme music corresponding to the identification information; store the theme music as playback information to obtain the first preset information base.

13. The information recommendation system according to claim 11, characterized by, The system further includes a fourth terminal configured to: in response to an object triggering a reading instruction of a preset book based on an electronic book display interface within a fifth preset time, display the preset book, and send a book type of the preset book to the server; The server is further configured to: acquire music corresponding to the book type; and store the music corresponding to the book type as playing information to obtain the first preset information base.

14. The information recommendation system according to claim 8, characterized by, The target mode is a sleep mode, and the first terminal is further configured to: in response to an object feature data collection instruction triggered by the object based on the information collection interface, collect second physiological data related to the sleep mode, and take the second physiological data as target feature data related to the sleep mode; The server is further configured to: receive the playing request, match the second physiological data with a preset sleep threshold range, when a duration in which the second physiological data is within the preset sleep threshold range is greater than a sixth preset time, take a sleep state corresponding to the preset sleep threshold range as a target sleep state, and take the target sleep state as the target correlation data analysis result; and configured to: acquire third target playing information corresponding to the target sleep state from a third preset information base, the third preset information base storing a mapping relationship between sleep states and playing information; and send the third target playing information to the second terminal. The second terminal is further configured to play the third target playing information.

15. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction or at least one program, the at least one instruction or the at least one program is loaded and executed by the processor to implement the object correlation data based information recommendation method of any one of claims 1 to 7.

16. An electronic device, comprising: The electronic device includes a processor and a memory, the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program is loaded and executed by the processor to implement the object correlation data based information recommendation method of any one of claims 1 to 7.

Images