A multimedia information processing method, device, equipment and storage medium

Abstract

The present application provides a multimedia information processing method, device, equipment and storage medium, the method comprising: when playing multimedia information, obtaining the current physiological data of the user; determining the data difference between the average physiological data and the current physiological data; the average physiological data is determined based on historical physiological data, the historical physiological data is data obtained within a first time period after receiving an authorization instruction to obtain physiological data; when the duration of the data difference being greater than a preset physiological threshold exceeds a preset time threshold, displaying an information prompt interface for closing the multimedia information; when within a second time period after displaying the information prompt interface, no playback instruction related to the multimedia information triggered by the information prompt interface is received, closing the playback of the multimedia information. The present application can improve the accuracy of automatically closing multimedia information that users watch while sleeping in a low-cost and simplified manner.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a multimedia information processing method, device, equipment and storage medium. Background Art

[0002] With the research and progress of artificial intelligence (AI), terminals have more functions besides making calls and sending and receiving short messages, such as playing multimedia information such as music and video.

[0003] Many users often fall asleep while watching or listening to multimedia information (such as videos, music, etc.), but after falling asleep, multimedia information such as music or videos will continue to play, which not only consumes the power of the terminal, but also may affect the user's sleep, even damage the user's physical health (for example, watching videos or listening to music through headphones for a long time will damage the user's eardrum, etc.) or wake the user from his sleep.

[0004] The prior art generally controls the playback of multimedia information by setting a time and setting the number of playbacks. However, after the set time or the set number of playbacks is reached, the user may not be asleep, which affects the user's viewing experience. The prior art also controls the multimedia playback by comparing the real-time heart rate with the previous real-time heart rate. When the real-time heart rate is greater than the threshold, the real-time collected heart rate will fluctuate, resulting in low control accuracy of the multimedia information. The prior art also controls the playback of multimedia information through the mutual cooperation between multiple sensors. However, the mutual cooperation between multiple sensors makes the communication mechanism between modules complex, the cost is high, and it is not universally adaptable. Summary of the invention

[0005] In order to improve the accuracy of automatically closing multimedia information that users watch when they sleep in a low-cost and simplified manner, the present application proposes a multimedia information processing method, device, equipment and storage medium.

[0006] On the one hand, the present application proposes a multimedia information processing method, the method comprising:

[0007] When multimedia information is being played, the user's current physiological data is obtained;

[0008] Determine a data difference between average physiological data and the current physiological data; the average physiological data is determined based on historical physiological data, the historical physiological data being data acquired within a first time period after receiving an authorization instruction to acquire physiological data;

[0009] When the duration of the data difference being greater than the preset physiological threshold exceeds the preset time threshold, an information prompt interface for closing the multimedia information is displayed;

[0010] When no play instruction related to the multimedia information triggered by the information prompt interface is received within a second time period after the information prompt interface is displayed, the play of the multimedia information is closed.

[0011] On the other hand, the present application proposes a multimedia information processing device, the device comprising:

[0012] The current physiological data acquisition module is used to obtain the user's current physiological data when playing multimedia information;

[0013] a data difference determination module, configured to determine a data difference between average physiological data and the current physiological data; the average physiological data is determined based on historical physiological data, the historical physiological data being data acquired within a first time period after receiving an authorization instruction to acquire physiological data;

[0014] An information prompt interface display module, used for displaying an information prompt interface for closing multimedia information when the duration of the data difference being greater than a preset physiological threshold exceeds a preset time threshold;

[0015] The multimedia information closing module is used to close the playing of the multimedia information when no playing instruction related to the multimedia information triggered by the information prompt interface is received within a second time period after the information prompt interface is displayed.

[0016] On the other hand, the present application proposes a device, comprising: a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the multimedia information processing method as described above.

[0017] On the other hand, the present application proposes a computer-readable storage medium, which stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the multimedia information processing method as described above.

[0018] The multimedia information processing method, device, equipment and storage medium proposed in the present application determine the average physiological data based on the data acquired within the first time period after receiving the authorization instruction to acquire the physiological data, compare the data difference between the average physiological data and the current physiological data and the size of the preset physiological threshold, and when the data difference is greater than the preset physiological threshold for a duration exceeding the preset time threshold, a message prompt interface pops up indicating that the multimedia information is about to be closed, and when no playback instruction related to the multimedia information triggered by the information prompt interface is received within the second time period after the information prompt interface is displayed, the playback of the multimedia information is closed. It can be seen that the present application needs to meet the following two conditions at the same time to close the playback of multimedia information: 1) the data difference between the average physiological data and the current physiological data is greater than the duration of the preset physiological threshold, exceeding the preset time threshold; 2) within the second time period after the information prompt interface is displayed, no playback instruction related to the multimedia information triggered by the information prompt interface is received. The present application determines whether the user is in a sleeping state only when the above two conditions are met at the same time. On the one hand, by comparing the current physiological data with the average physiological data determined based on the historical physiological data, it is possible to avoid misjudgment of the user's sleeping state caused by direct comparison or real-time comparison of the physiological data collected at two previous and subsequent time points (for example, if the user has a short-term "false sleep phenomenon" such as "napping" or "taking a nap", the real-time comparison of the physiological data will lead to misjudgment of the sleeping state), thereby improving the accuracy and reliability of judging whether the user is in a sleeping state. On the other hand, when it is determined that the data difference is greater than the preset physiological threshold and the duration exceeds the preset time threshold, it is not immediately judged that the user is in a sleeping state, but a message prompt interface for closing the multimedia information is displayed, and whether the user is in a sleeping state is further judged based on whether the user triggers the operation in the information prompt interface, thereby further avoiding the user from being misjudged. On the other hand, the user only needs to have a favorite smart device (such as a bracelet, etc.) that can monitor physiological data and connect the smart device to the user terminal, so as to control the playback of multimedia information in the user terminal in a simple and low-cost manner, and the popularity is relatively high. On the other hand, the present application automatically turns off the multimedia information after judging that the user is in a sleeping state, which can not only avoid the influence of the multimedia information that continues to be played on the user when the user is in a sleeping state, but also improve the user's sleep quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 It is a flowchart of a multimedia information processing method provided in an embodiment of the present application.

[0021] Figure 2 It is a structural diagram of an information authorization interface provided in an embodiment of the present application.

[0022] Figure 3 It is a flowchart of another multimedia information processing method provided in an embodiment of the present application.

[0023] Figure 4 It is a structural schematic diagram of an information prompt interface for closing multimedia information provided in an embodiment of the present application.

[0024] Figure 5 It is a flowchart of another multimedia information processing method provided in an embodiment of the present application.

[0025] Figure 6 It is a structural schematic diagram of a threshold setting interface provided in an embodiment of the present application.

[0026] Figure 7 It is a structural schematic diagram of a mode selection interface provided in an embodiment of the present application.

[0027] Figure 8 It is an optional structural diagram of the blockchain system provided in the embodiment of the present application.

[0028] Fig. 9 It is an optional schematic diagram of the block structure provided in the embodiment of the present application.

[0029] Fig.10 It is a flowchart of another multimedia information processing method provided in an embodiment of the present application.

[0030] Fig.11 It is a structural schematic diagram of a multimedia information processing device provided in an embodiment of the present application.

[0031] Fig.12 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] With the research and progress of artificial intelligence (AI), AI has been studied and applied in many fields. AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence.

[0033] Specifically, the solution provided in the embodiments of the present application relates to computer vision technology (CV) of artificial intelligence. CV refers to machine vision such as using cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instrument detection. CV generally includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning and map construction, and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0034] Specifically, the multimedia information processing method provided in the embodiments of the present application relates to video processing technology in CV, etc.

[0035] Specifically, the technical solutions provided in the embodiments of the present application are illustrated by the following embodiments.

[0036] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] Figure 1It is a flowchart of a multimedia information processing method provided by an embodiment of the present application. This specification provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 1 As shown, the method may include:

[0039] S101. When multimedia information is being played, obtain current physiological data of the user.

[0040] In an embodiment of the present application, a user can watch or listen to multimedia information through a terminal, and the terminal can obtain the user's current physiological data, and automatically close the multimedia information currently watched or listened to by the user when it is determined that the user is in a sleep state based on the current physiological data.

[0041] In the embodiment of the present application, the terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, etc. with a multimedia information playback function, but is not limited thereto.

[0042] In a feasible embodiment, the current physiological data can be collected by a smart wearable device carried by the user. For example, the current physiological data can be the data collected by the smart wearable device in a unit time, and the unit time can be 20 seconds, 30 seconds, 45 seconds or 1 minute, etc. The smart wearable device can be a device that the user likes and can collect physiological data related to the sleep state, including but not limited to smart bracelets, smart watches, smart glasses, smart headbands, etc. In the embodiment of the present application, the physiological data is continuously collected through a device that the user likes and can collect physiological data related to the sleep state, which not only has a low cost, but also has a wide range of adaptability, avoiding the defects of complex communication mechanism, high cost and difficulty in popularization caused by the use of more modules.

[0043] In practical applications, in order for the terminal to obtain the physiological data collected by the smart wearable device, the user can pre-establish a connection relationship between the smart wearable device and the terminal, and the connection relationship is preferably a short-range wireless communication connection, which includes but is not limited to Bluetooth, infrared, ZigBee, near-field communication, ultra-wideband communication, etc. Of course, the connection relationship can also be a wired communication connection.

[0044] In actual applications, in order for the terminal to obtain the physiological data collected by the smart wearable device, the user can also pre-authorize the terminal so that the terminal has the authority to obtain the physiological data collected by the smart wearable device. The authorization process can be: after establishing a short-range wireless communication connection with the smart wearable device, the terminal can pop up a pop-up window such as Figure 2 The information authorization interface shown in the figure may include the option of "whether to allow the acquisition of data collected by the smart wearable device". When the user clicks the "Yes" option, the authorization of the terminal is completed. When the user clicks the "No" option, the authorization of the terminal is not completed. After the terminal is authorized, the physiological data collected by it can be obtained from the smart wearable device in real time through a short-range wireless communication connection in S101. Only authorized terminals have the right to obtain the physiological data collected by the smart wearable device, and unauthorized terminals have no right to obtain it, which can effectively improve the accuracy, credibility and flexibility of physiological data acquisition.

[0045] In practical applications, in addition to authorizing the terminal, the smart wearable device can also be authorized to have the authority to send the collected physiological data to the terminal.

[0046] It should be noted that Figure 2 Just an example.

[0047] In the embodiment of the present application, the current physiological data and the physiological data mentioned subsequently, such as average physiological data, etc., can all be physiological data related to the sleep state. The data related to the sleep state includes but is not limited to heart rate data, breathing data, blood oxygen data, blood pressure data, etc.

[0048] S103. Determine the data difference between the average physiological data and the current physiological data; the average physiological data is determined based on the historical physiological data, and the historical physiological data is the data acquired within the first time period after receiving the authorization instruction to acquire the physiological data.

[0049] In an embodiment of the present application, after the terminal obtains the current physiological data of the user from the smart wearable device, it can first determine whether the average physiological data is already stored in the storage system of the terminal. If the average physiological data is already stored, it indicates that this is not the first time that the terminal obtains physiological data from the smart wearable device. The current physiological data can be directly compared with the average physiological data to determine the data difference between the average physiological data and the current physiological data. If the average physiological data does not exist in the storage system, it indicates that this is the first time that the terminal obtains physiological data from the smart wearable device, so it is necessary to calculate the average physiological data. The average physiological data can be determined based on the data obtained within the first time period (for example, three days, seven days, ten days, fifteen days, etc.) after the user authorizes the terminal to obtain physiological data. Specifically, Figure 3As shown, before S101, the method may further include a step of acquiring average physiological data, and the acquiring average physiological data may include:

[0050] S011. After establishing a short-range wireless communication connection with the smart wearable device, an information authorization interface is displayed.

[0051] S013. In response to the authorization instruction triggered in the information authorization interface, a plurality of historical physiological data are received from the smart wearable device via a short-range wireless communication connection within a first time period.

[0052] S015. Take the average value of multiple historical physiological data as the average physiological data.

[0053] In practical applications, after the average physiological data is obtained, the average physiological data may be stored in a storage system for subsequent use.

[0054] In actual applications, the average physiological data is determined based on the physiological data of all states (sleep state and non-sleep state, etc.) obtained within the first time period after the user authorizes the terminal to obtain the physiological data. Of course, in order to further improve the accuracy of the average physiological data, it is also possible to only obtain the physiological data of the user in the non-sleep state within the first time period, and determine the average physiological data based on the physiological data of the non-sleep state.

[0055] In the embodiment of the present application, since the average physiological data is determined based on the data obtained within the first time period after the user authorizes the terminal to obtain the physiological data, the average physiological data can accurately reflect the user's physiological state. Using the average physiological data with higher accuracy as a comparison benchmark for the current physiological data can further improve the accuracy and credibility of the subsequent automatic shutdown of the playback of the multimedia information.

[0056] S105. When the duration of the data difference being greater than the preset physiological threshold exceeds the preset time threshold, an information prompt interface for closing the multimedia information is displayed.

[0057] In actual applications, the duration can be 3-8 minutes. If the data difference is continuously greater than the preset time threshold within 3-8 minutes, a pop-up message will appear. Figure 4 The information prompt interface for closing the multimedia message and pausing the multimedia message may display "the multimedia message will be closed soon", or "the multimedia message will be closed in XX seconds", and "whether to continue playing the multimedia message".

[0058] It should be noted that Figure 4 Just an example.

[0059] S107. When no playback instruction related to the multimedia information triggered by the information prompt interface is received within a second time period after the information prompt interface is displayed, it is determined that the user is in a sleeping state, and the playback of the multimedia information is closed.

[0060] In actual applications, the playback instructions related to the multimedia information include a continue playback instruction or a non-continue playback instruction. If the user does not select "Yes" or "No" in the information prompt interface within the second time period (for example, 5-10 seconds) after the information prompt interface pops up, that is, the user does not make any feedback in the information prompt interface, that is, the terminal does not receive the continue playback instruction or the non-continue playback instruction triggered by the information prompt interface, it indicates that the user is in sleep state at this time, and the terminal can automatically close the playback of the multimedia information immediately.

[0061] In the embodiment of the present application, when the duration of the data difference being greater than the preset physiological threshold exceeds the preset time threshold, the playback of the multimedia information is not immediately closed, but a second judgment is made on this basis, that is, whether the instruction of continuing to play or not continuing to play triggered by the user on the information prompt interface is received within the second time period after the information prompt interface is popped up. After no feedback triggered by the user is received, it is determined that the user is in a sleep state and the playback of the multimedia information is automatically closed. It can be seen that the automatic closing of the playback of the multimedia information in the embodiment of the present application is the result of the double guarantee of "the duration of the data difference between the average physiological data and the current physiological data being greater than the preset physiological threshold exceeds the preset time threshold" and "no playback instruction related to the multimedia information triggered by the information prompt interface is received within the second time period". On the one hand, comparing the current physiological data with the average physiological data determined based on the historical physiological data can avoid the misjudgment of the user's sleep state caused by direct comparison or real-time comparison of the physiological data collected at the two previous and subsequent time points (for example, if the user has a short moment of "false sleep phenomenon" such as "dozing" or "taking a nap" , real-time comparison of physiological data will lead to misjudgment of sleep status), thereby improving the accuracy and reliability of judging whether the user is in a sleep state; on the other hand, in the second time period, whether the user is in a sleep state is further judged based on whether the user triggers an operation in the information prompt interface, which can further avoid misjudgment of the user's sleep state (for example, if the user operation is received within the second time, the user may be in a non-sleep state. At this time, the operation triggered by the user in the information prompt interface can avoid the incorrect closing of multimedia information due to misjudgment of the user's sleep state), thereby further improving the accuracy, reliability, and user experience of judging whether the user is in a sleep state, and having high robustness.

[0062] In practical applications, such as Figure 5As shown, if the user selects "Yes" in the information prompt interface within the second time period after the information prompt interface pops up, that is, when the terminal receives the instruction to continue playing triggered based on the information prompt interface, the method may also include:

[0063] S201. When a continue playing instruction triggered by the information prompt interface is received within the second time period, the distance information between the smart wearable device and the smart wearable device is determined.

[0064] S203. When the distance information is less than or equal to the preset distance threshold, continue to play the multimedia information, and repeat the steps of obtaining the current physiological data, determining the data difference, and displaying the information prompt interface.

[0065] S205. When the distance information is greater than the preset distance threshold, continue to play the multimedia information and interrupt the process of acquiring the current physiological data.

[0066] In actual application, if within the second time period, the terminal receives a continue playing instruction triggered by the information prompt interface, and the distance information is less than or equal to the preset distance threshold, it can be considered that the user is not in a sleep state at this time, and the terminal can continue to play the multimedia information and repeat S101-S105. If the terminal receives a continue playing instruction triggered by the information prompt interface, and the distance information is greater than the preset distance threshold, it can be considered that the user is in a sleep state at this time, but the terminal is used by other users other than the user, and the multimedia information can be continued to be played for other users to obtain the multimedia information, but because the user is in a sleep state at this time, the terminal is used by other users, so the process of obtaining the current physiological data can be interrupted to avoid affecting the viewing experience of other users. If within the second time period, the terminal receives a non-continue playing instruction triggered by the information prompt interface, it can be considered that the user is not in a sleep state at this time, and the terminal can stop continuing to play the multimedia information.

[0067] In an embodiment of the present application, when a continue playback instruction triggered by an information prompt interface is received, the judgment of the distance information between the terminal and the smart wearable device can reduce the error of automatically shutting down the playback of multimedia information when the user is in a sleep state, and further improve the accuracy and feasibility of automatically shutting down the playback of multimedia information when the user is in a sleep state.

[0068] In the embodiment of the present application, in order to accurately realize the automatic closing of the multimedia information being watched when the user is asleep, the various thresholds mentioned above may be preset, that is, before S101, the method may further include:

[0069] S021. Display the threshold setting interface.

[0070] S023. In response to the threshold setting instruction triggered in the threshold setting interface, the preset physiological threshold, the preset time threshold and the preset distance threshold are set.

[0071] In practical applications, the terminal can also provide Figure 6 The threshold setting interface shown may include a preset physiological threshold setting option, a preset time threshold setting option, a preset distance setting option, etc. When the user clicks the preset physiological threshold setting option, the preset time threshold setting option or the preset distance setting option, that is, when the terminal receives the threshold setting instruction triggered by the threshold setting interface, the terminal can set each threshold according to its actual situation and actual usage scenarios, with high flexibility and good user experience.

[0072] In order to improve the scope of use, general adaptability and user experience of the embodiments of the present application, different modes may be set for the user to choose, that is, before S101, the method may further include:

[0073] S031. Display mode selection interface.

[0074] S033. In response to a sleep mode instruction triggered in the mode selection interface, execute the steps of obtaining the user's current physiological data, determining the data difference, displaying an information prompt interface, and turning off the playback of multimedia information; or, in response to a health mode instruction triggered in the mode selection interface, obtain the user's current physiological data and display the current physiological data in real time on the multimedia information playback interface.

[0075] In practical applications, the terminal can also provide Figure 7 The mode selection interface shown in the figure can be set with multiple modes, including but not limited to sleep mode, health mode, etc. If the user selects the sleep mode in the mode selection interface, that is, the terminal receives the sleep mode instruction triggered based on the mode selection interface, it means that the user wants the terminal to automatically close the multimedia information viewed when sleeping, and S101-S107 can be executed. If the user selects the health mode in the mode selection interface, that is, the terminal receives the health mode instruction triggered based on the mode selection interface, it means that the user wants to monitor himself in real time, and the terminal can obtain the physiological data collected by the smart wearable device, and display the physiological data in real time on the multimedia information playback interface, so that the user can obtain his own physical health status in real time, thereby further improving the user experience.

[0076] In a feasible embodiment, the current physiological data in S101, the average physiological data in S103, the preset physiological threshold and preset time threshold in S105, the preset distance threshold in S203, etc. can all be stored in the blockchain system. Figure 8 , Figure 8 The figure shows an optional structural diagram of the blockchain system provided by the embodiment of the present application, wherein a peer-to-peer (P2P) network is formed between multiple nodes, and the P2P protocol is an application layer protocol running on the Transmission Control Protocol (TCP). In the blockchain system, any machine such as a server or a terminal can join and become a node, and the node includes a hardware layer, an intermediate layer, an operating system layer, and an application layer.

[0077] See also Figure 8 The functions of each node in the blockchain system shown include:

[0078] 1) Routing: a basic function of a node, used to support communication between nodes.

[0079] In addition to the routing function, the node can also have the following functions:

[0080] 2) Applications are deployed in the blockchain to implement specific businesses based on actual business needs, record data related to the implementation of functions to form record data, carry digital signatures in the record data to indicate the source of the task data, and send the record data to other nodes in the blockchain system for other nodes to add the record data to a temporary block when they successfully verify the source and integrity of the record data.

[0081] 3) Blockchain, including a series of blocks that are connected to each other in the order of their generation. Once a new block is added to the blockchain, it will not be removed. The block records the record data submitted by the nodes in the blockchain system.

[0082] See also Fig. 9 , Fig. 9 An optional schematic diagram of the block structure (Block Structure) provided for the embodiment of the present application, each block includes the hash value of the transaction record stored in this block (the hash value of this block) and the hash value of the previous block, and each block is connected by the hash value to form a blockchain. In addition, the block can also include information such as the timestamp when the block was generated. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains relevant information for verifying the validity of its information (anti-counterfeiting) and generating the next block.

[0083] Fig.10 FIG. 1 is a flow chart of a multimedia information processing method provided by an embodiment of the present application in a scenario, wherein the physiological data in the scenario may be heart rate data, and the multimedia information may be a video. Specifically, Fig.10As shown, the embodiment of the present application determines the average heart rate data through the heart rate data obtained within the first time period after receiving the authorization instruction to obtain the physiological data, and compares the data difference between the average heart rate data and the current heart rate data with the preset physiological threshold. When the duration of the data difference being greater than the preset physiological threshold exceeds the preset time threshold, and when the user does not trigger the play instruction of continuing to play the video in the information prompt interface within the second time period after the information prompt interface is displayed, it is determined that the user is in a sleeping state at this time, and the video is automatically closed (for example, the video player playing the video is closed). The multimedia information processing method provided by the embodiment of the present application has the following beneficial effects:

[0084] 1) The automatic shutdown of multimedia information playback in the embodiment of the present application is the result of the dual guarantees of "the duration of the data difference between the average physiological data and the current physiological data being greater than the preset physiological threshold exceeds the preset time threshold" and "no video-related playback instructions triggered by the information prompt interface are received during the second time period". On the one hand, comparing the current physiological data with the average physiological data determined based on the historical physiological data can avoid misjudgment of the user's sleep state caused by direct comparison or real-time comparison of physiological data collected at two time points before and after (for example, if the user has a short "false sleep phenomenon" such as "dozing" or "taking a nap", real-time comparison of physiological data will lead to misjudgment of the sleep state), thereby improving the ability to judge whether the user is in a state of sleep. On the other hand, when it is determined that the data difference is greater than the preset physiological threshold and the duration exceeds the preset time threshold, the user is not immediately judged to be in a sleep state, but an information prompt interface for closing the multimedia information is displayed, and whether the user is in a sleep state is further judged based on whether the user triggers an operation in the information prompt interface. This can further avoid misjudgment of the user's sleep state (for example, if the user operation is received within the second time, the user may be in a non-sleep state. At this time, the operation triggered by the user in the information prompt interface can avoid the mistaken closing of multimedia information due to misjudgment of the user's sleep state), thereby further improving the accuracy and reliability of judging whether the user is in a sleep state and user experience, and has high robustness.

[0085] 2) The embodiment of the present application only requires the user to have a favorite smart wearable device (such as a smart bracelet, etc.) that can monitor physiological data (such as heart rate), authorize the smart wearable device and the terminal, and realize communication between the terminal and the smart wearable device through short-range wireless communication technology. In this way, the currently watched video can be automatically closed, avoiding the complex inter-module communication mechanism caused by the use of more modules. It has a simple structure, low cost, wide adaptability and high popularity.

[0086] 3) After determining that the user is in a sleeping state, the embodiment of the present application automatically turns off the multimedia information, which can not only avoid the impact of the multimedia information that continues to be played on the user when the user is in a sleeping state, thereby improving the user's sleep quality, but also save terminal power.

[0087] like Fig.11 As shown, an embodiment of the present application provides a multimedia information processing device, which may include:

[0088] The current physiological data acquisition module 301 is used to acquire the current physiological data of the user when multimedia information is being played.

[0089] Specifically, the current physiological data is collected by a smart wearable device carried by the user, and the current physiological data acquisition module 301 can be further used to:

[0090] Receive current physiological data sent by smart wearable devices via short-range wireless communication connection.

[0091] The data difference determination module 303 can be used to determine the data difference between the average physiological data and the current physiological data; the average physiological data is determined based on the historical physiological data, and the historical physiological data is the data obtained within the first time period after receiving the authorization instruction to obtain the physiological data.

[0092] The information prompt interface display module 305 can be used to display an information prompt interface for closing multimedia information when the duration of the data difference being greater than a preset physiological threshold exceeds a preset time threshold.

[0093] The multimedia information closing module 307 may be used to determine that the user is in a sleeping state and close the playing of the multimedia information when no playing instruction related to the multimedia information triggered by the information prompt interface is received within a second time period after the information prompt interface is displayed.

[0094] In an embodiment of the present application, the playback instructions related to the multimedia information include a continue playback instruction or a non-continue playback instruction, then the multimedia information closing module 307 can also be used to close the playback of the multimedia information when no continue playback instruction or non-continue playback instruction triggered by the information prompt interface is received within the second time period.

[0095] In the embodiment of the present application, the device may further include an average physiological data acquisition module, and the average physiological data acquisition module may include:

[0096] The information authorization interface display unit can be used to display the information authorization interface after establishing a short-range wireless communication connection with the smart wearable device.

[0097] The authorization instruction response unit can be used to respond to the authorization instruction triggered in the information authorization interface, and receive multiple historical physiological data sent by the smart wearable device through a short-range wireless communication connection within a first time period.

[0098] The average physiological data determination unit may be configured to take an average value of a plurality of historical physiological data as the average physiological data.

[0099] In the embodiment of the present application, the device may further include:

[0100] The pause module can be used to pause the playing of multimedia information after displaying the information prompt interface for closing the multimedia information when the duration of the data difference being greater than the preset physiological threshold exceeds the preset time threshold.

[0101] In the embodiment of the present application, the device may further include:

[0102] The distance information determination module can be used to determine the distance information between the smart wearable device and the smart wearable device when the continue playing instruction triggered by the information prompt interface is received within the second time period.

[0103] The repeat module can be used to continue playing the multimedia information and repeat the steps of acquiring the current physiological data, determining the data difference, and displaying the information prompt interface when the distance information is less than or equal to the preset distance threshold.

[0104] The interruption module can be used to continue playing the multimedia information and interrupt the process of obtaining the current physiological data when the distance information is greater than a preset distance threshold.

[0105] In the embodiment of the present application, the device may further include:

[0106] The threshold setting interface display module can be used to display the threshold setting interface.

[0107] The threshold setting instruction response module can be used to set the preset physiological threshold, the preset time threshold and the preset distance threshold in response to the threshold setting instruction triggered on the threshold setting interface.

[0108] In the embodiment of the present application, the device may further include:

[0109] The mode selection interface display module can be used to display the mode selection interface.

[0110] The sleep or health mode instruction response module can be used to respond to the sleep mode instruction triggered in the mode selection interface, execute the steps of obtaining the user's current physiological data, determining the data difference, displaying the information prompt interface, and closing the playback of multimedia information; or, respond to the health mode instruction triggered in the mode selection interface, obtain the user's current physiological data, and display the current physiological data in real time on the multimedia information playback interface.

[0111] An embodiment of the present application also provides an electronic device for multimedia information processing, the electronic device comprising a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement the multimedia information processing method provided in the above method embodiment.

[0112] An embodiment of the present application also provides a computer-readable storage medium, which can be set in a terminal to store at least one instruction or at least one program related to a multimedia information processing method in a method embodiment, and the at least one instruction or at least one program is loaded and executed by a processor to implement the multimedia information processing method provided in the above method embodiment.

[0113] Optionally, in the embodiment of this specification, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the above storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0114] The memory described in the embodiments of this specification can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, applications required for functions, etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0115] The multimedia information processing method provided in the embodiments of the present application can be executed in a terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Fig.121 is a hardware structure block diagram of a server of a multimedia information processing method provided in an embodiment of the present application. Fig.12 As shown, the server 400 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 410 (the processor 410 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 430 for storing data, and one or more storage media 420 (such as one or more mass storage devices) for storing application programs 423 or data 422. Among them, the memory 430 and the storage medium 420 can be short-term storage or permanent storage. The program stored in the storage medium 420 may include one or more modules, each of which may include a series of instruction operations on the server. Furthermore, the central processing unit 410 can be configured to communicate with the storage medium 420 and execute a series of instruction operations in the storage medium 420 on the server 400. The server 400 may also include one or more power supplies 460, one or more wired or wireless network interfaces 450, one or more input and output interfaces 440, and / or, one or more operating systems 421, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM etc.

[0116] The input / output interface 440 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the server 400. In one example, the input / output interface 440 includes a network adapter (Network Interface Controller, NIC), which may be connected to other network devices via a base station so as to communicate with the Internet. In one example, the input / output interface 440 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0117] It can be understood by those skilled in the art that Fig.12 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig.12 More or fewer components as shown, or with Fig.12 Different configurations are shown.

[0118] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of this specification are described. Other embodiments are within the scope of the attached claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0119] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and server embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0120] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0121] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A multimedia information processing method, characterized in that: The method comprises: When multimedia information is being played, the user's current physiological data is obtained; the current physiological data is collected by a smart wearable device carried by the user; Determine a data difference between average physiological data and the current physiological data; the average physiological data is determined based on historical physiological data, the historical physiological data being data acquired within a first time period after receiving an authorization instruction to acquire physiological data; When the duration of the data difference being greater than the preset physiological threshold exceeds the preset time threshold, an information prompt interface for closing the multimedia information is displayed; When no continue playback instruction or non-continue playback instruction triggered by the information prompt interface is received within a second time period after the information prompt interface is displayed, it is determined that the user is in a sleep state and the playback of the multimedia information is turned off; when the continue playback instruction triggered by the information prompt interface is received within the second time period, the distance information between the user and the smart wearable device is determined; when the distance information is less than or equal to a preset distance threshold, the multimedia information is continued to be played, and the steps of obtaining the current physiological data, determining the data difference, and displaying the information prompt interface are repeated; when the distance information is greater than the preset distance threshold, the multimedia information is continued to be played, and the process of obtaining the current physiological data is interrupted.

2. The method according to claim 1, characterized in that The step of obtaining the current physiological data of the user during multimedia information playback includes: Receive the current physiological data sent by the smart wearable device via a short-range wireless communication connection.

3. The method according to claim 2, characterized in that The method further comprises the step of acquiring the average physiological data, wherein acquiring the average physiological data comprises: After establishing the short-range wireless communication connection with the smart wearable device, displaying an information authorization interface; In response to the authorization instruction triggered in the information authorization interface, receiving a plurality of historical physiological data sent by the smart wearable device through the short-range wireless communication connection within the first time period; An average value of the plurality of historical physiological data is used as the average physiological data.

4. The method according to claim 2, characterized in that: After displaying an information prompt interface for closing multimedia information when the duration of the data difference being greater than the preset physiological threshold exceeds the preset time threshold, the method further includes: Pause playing of the multimedia information.

5. The method according to claim 1, characterized in that When playing multimedia information, before obtaining the current physiological data of the user, the method further includes: Display the threshold setting interface; In response to a threshold setting instruction triggered in the threshold setting interface, the preset physiological threshold, the preset time threshold and the preset distance threshold are set.

6. The method according to claim 1, characterized in that When playing multimedia information, before obtaining the current physiological data of the user, the method further includes: Display mode selection interface; In response to a sleep mode instruction triggered in the mode selection interface, the steps of obtaining the user's current physiological data, determining the data difference, displaying the information prompt interface, and closing the playback of the multimedia information are executed; or, in response to a health mode instruction triggered in the mode selection interface, the user's current physiological data is obtained, and the current physiological data is displayed in real time on the playback interface of the multimedia information.

7. A multimedia information processing device, characterized in that: The device comprises: A current physiological data acquisition module is used to acquire the user's current physiological data when multimedia information is played; the current physiological data is collected by a smart wearable device carried by the user; a data difference determination module, configured to determine a data difference between average physiological data and the current physiological data; the average physiological data is determined based on historical physiological data, the historical physiological data being data acquired within a first time period after receiving an authorization instruction to acquire physiological data; An information prompt interface display module, used for displaying an information prompt interface for closing multimedia information when the duration of the data difference being greater than a preset physiological threshold exceeds a preset time threshold; A multimedia information closing module, configured to determine that the user is in a sleeping state and close the playing of the multimedia information when no continue playing instruction or non-continue playing instruction triggered by the information prompt interface is received within a second time period after the information prompt interface is displayed; a distance information determination module, configured to determine the distance information between the smart wearable device and the smart wearable device when the continue playing instruction triggered by the information prompt interface is received within the second time period; A repeating module, configured to continue playing the multimedia information and repeat the steps of acquiring the current physiological data, determining the data difference, and displaying the information prompt interface when the distance information is less than or equal to a preset distance threshold; The interruption module is used to continue playing the multimedia information and interrupt the process of obtaining the current physiological data when the distance information is greater than the preset distance threshold.

8. The device according to claim 7, characterized in that The current physiological data acquisition module is used to receive the current physiological data sent by the smart wearable device via a short-range wireless communication connection.

9. The device according to claim 8, characterized in that The device further comprises an average physiological data acquisition module, wherein the average physiological data acquisition module comprises: An information authorization interface display unit, configured to display an information authorization interface after establishing the short-range wireless communication connection with the smart wearable device; an authorization instruction response unit, configured to respond to the authorization instruction triggered in the information authorization interface, and receive a plurality of historical physiological data sent by the smart wearable device through the short-range wireless communication connection within the first time period; The average physiological data determination unit is used to use the average value of the multiple historical physiological data as the average physiological data.

10. The device according to claim 8, characterized in that The device also includes: The pause module is used to pause the playing of the multimedia information.

11. The device according to claim 7, characterized in that The device also includes: A threshold setting interface display module, used to display a threshold setting interface; The threshold setting instruction response module is used to set the preset physiological threshold, the preset time threshold and the preset distance threshold in response to the threshold setting instruction triggered in the threshold setting interface.

12. The device according to claim 7, characterized in that The device also includes: A mode selection interface display module, used to display a mode selection interface; A sleep or health mode instruction response module is used to respond to a sleep mode instruction triggered in the mode selection interface to execute the steps of obtaining the user's current physiological data, determining the data difference, displaying the information prompt interface, and closing the playback of the multimedia information; or, in response to a health mode instruction triggered in the mode selection interface, obtain the user's current physiological data and display the current physiological data in real time on the playback interface of the multimedia information.

13. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the multimedia information processing method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the multimedia information processing method according to any one of claims 1 to 6.

Images