Video reminding method and device

Abstract

The embodiment of the invention provides a video reminding method and device, computer equipment, a medium and a program product. Relates to the technical field of computers. The method comprises the following steps: monitoring a current playing time point in real time in a video playing process; under the condition that the distance between the current playing time point and the starting time point of the target video clip is a preset duration, collecting a user image; performing face recognition on the user image, and performing reminding operation on a user by adopting a first reminding mode under the condition of recognizing that the user image contains a face image; and under the condition that it is recognized that the user image does not contain the face image, adopting a second reminding mode to carry out reminding operation on the user. According to the technical scheme of the embodiment of the invention, the user can be prevented from missing the target video clip, and the watching experience and satisfaction of the user are improved.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of computer technology, and in particular, to a video reminder method, device, computer device, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of computer technology, the application of multimedia has become more and more extensive, and various videos have emerged on the network. When users watch videos, they often miss high-heat segments or wonderful video segments in the videos for various reasons (such as distraction, multitasking, etc.).

[0003] Since these segments are usually the essence of the video content, when users miss watching these segments, it will significantly affect the user's viewing experience and satisfaction.

[0004] It should be noted that the above content is not necessarily the prior art and is not used to limit the patent protection scope of the present application. Summary of the Invention

[0005] Embodiments of the present application provide a video reminder method, device, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the above technical problems.

[0006] One aspect of the embodiments of the present application provides a video reminder method, and the method includes: During the video playback process, the current playback time point is monitored in real time; When the distance between the current playback time point and the start time point of the target video segment is a preset duration, a user image is collected; Face recognition is performed on the user image, and when it is recognized that the user image contains a face image, a first reminder method is used to perform a reminder operation on the user; When it is recognized that the user image does not contain a face image, a second reminder method is used to perform a reminder operation on the user.

[0007] Optionally, the first reminder method includes a reminder means and a reminder amplitude. When it is recognized that the user image contains a face image, using the first reminder method to perform a reminder operation on the user includes: When it is recognized that the user image contains a face image, key feature points of the face are obtained; Based on the key feature points, it is determined whether the user's line of sight is on the screen for playing the video; When the user's line of sight is on the screen, the first reminder means and the first reminder amplitude for performing a reminder operation on the user are determined; Perform a reminder operation on the user using a determined first reminder means and a first reminder amplitude.

[0008] Optionally, the method further includes: When the user's line of sight is outside the screen, determine a second reminder means and a second reminder amplitude for performing a reminder operation on the user; Perform a reminder operation on the user using the determined second reminder means and the second reminder amplitude.

[0009] Optionally, determining whether the user's line of sight is on the screen for playing the video based on the key feature points includes: Determine the user's line of sight angle based on the key feature points; Determine whether the user's line of sight is on the screen for playing the video based on the line of sight angle.

[0010] Optionally, the key feature points include the center point of the left eye, the center point of the right eye, the position point of the nose tip, and the center point of the mouth. Determining whether the user's line of sight is on the screen for playing the video based on the key feature points includes: Connect the center point of the left eye, the center point of the right eye, and the center point of the mouth to form a triangle; Determine whether the position of the nose tip is inside the triangle; When the position of the nose tip is inside the triangle, calculate a first distance between the center point of the left eye and the center point of the right eye; Obtain the target center point of the line connecting the center point of the left eye and the center point of the right eye; Calculate a second distance between the target center point and the center point of the mouth; Calculate the ratio of the first distance to the second distance; When the ratio is within a preset range, determine that the user's line of sight is on the screen.

[0011] Optionally, determining whether the user's line of sight is on the screen for playing the video based on the key feature points further includes: When the position of the nose tip is outside the triangle, determine that the user's line of sight is outside the screen; or When the ratio is outside the preset range, determine that the user's line of sight is outside the screen.

[0012] Optionally, when the user's line of sight is on the screen, determining a first reminder means and a first reminder amplitude for performing a reminder operation on the user includes: When the user's line of sight is on the screen, determine the user's line of sight angle based on the key feature points; Determine the first reminder amplitude based on the line-of-sight angle; Determine the first reminder means based on the video playback platform.

[0013] Optionally, the method further includes: Determine the target video segment in the video.

[0014] Another aspect of the embodiments of the present application provides a video reminder device, and the device includes: A monitoring module, configured to monitor the current playback time point in real time during video playback; An acquisition module, configured to acquire a user image when the distance between the current playback time point and the start time point of the target video segment is a preset duration; A first reminder module, configured to perform face recognition on the user image, and when it is recognized that the user image contains a face image, perform a reminder operation on the user in a first reminder manner; A second reminder module, configured to perform a reminder operation on the user in a second reminder manner when it is recognized that the user image does not contain a face image.

[0015] Another aspect of the embodiments of the present application provides a computer device, including: At least one processor; and A memory communicatively connected to the at least one processor; Wherein: the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0016] Another aspect of the embodiments of the present application provides a computer-readable storage medium, in which computer instructions are stored, and when the computer instructions are executed by a processor, the method as described above is implemented.

[0017] Another aspect of the embodiments of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method as described above is implemented.

[0018] The embodiments of the present application adopting the above technical solutions may include the following advantages: During the video playback process, the current playback time point is monitored in real time, and when the current playback time point is about to reach the start time point of the target video segment (high - popularity segment, exciting video segment), a user image is collected. When a face image is included in the collected user image, it indicates that the user has not left. At this time, the first reminder method (for example, the video playback frame flashes, the mobile phone vibrates, etc.) is used to perform a reminder operation on the user to remind the user that the target video segment is about to be played, so as to prevent the user from missing the target video segment. When a face image is not included in the collected user image, at this time, the second reminder method (for example, voice broadcast, beeping sound, etc.) is used to perform a reminder operation on the user to remind the user that the target video segment is about to be played, so as to prevent the user from missing the target video segment. In this embodiment, by using different reminder methods based on different scenarios to perform reminder operations on the user, it can be ensured that the user can receive reminders under any circumstances, preventing the user from missing the target video segment and improving the user's viewing experience and satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings exemplarily show embodiments and form a part of the specification, and are used together with the written description of the specification to explain the exemplary embodiments. The shown embodiments are for illustrative purposes only and do not limit the scope of the claims. In all the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0020] Figure 1 Schematically shows the operating environment diagram of the video reminder method according to Embodiment 1 of the present application; Figure 2 Schematically shows the flowchart of the video reminder method according to Embodiment 1 of the present application; Figure 3 Schematically shows Figure 1 the sub - step flowchart of step S204; Figure 4 Schematically shows the detailed flowchart of the step of determining whether the user's line of sight is on the screen for playing the video based on the key feature points; Figure 5 Schematically shows the detailed flowchart of the step of determining whether the user's line of sight is on the screen for playing the video based on the key feature points; Figure 6 and Figure 7 Schematically shows a schematic diagram of a human face; Figure 8 Schematically shows the detailed flowchart of the step of determining the first reminder means and the first reminder amplitude for performing a reminder operation on the user when the user's line of sight is on the screen; Figure 9Schematically shows a block diagram of a video reminder device according to Embodiment 2 of the present application; and Figure 10 Schematically shows a schematic diagram of the hardware architecture of a computer device according to Embodiment 3 of the present application. Detailed implementation manners

[0021] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0022] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions between the various embodiments may be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement them. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present application.

[0023] In the description of the present application, it should be understood that the numerical labels before the steps do not identify the order of execution of the steps, but are only used to facilitate the description of the present application and distinguish each step, and thus cannot be understood as a limitation to the present application.

[0024] First, provide the term explanations involved in the present application: Face recognition algorithm: It is an algorithm that uses deep learning, support vector machine (SVM) and other algorithms to identify the key feature points of a face. Through the algorithm, the positions of multiple features of the face can be identified, including the positions and sizes of key parts such as the face contour, eyes, eyebrows, nose, and mouth.

[0025] To facilitate the understanding of the technical solutions provided by the embodiments of the present application by those skilled in the art, the related technologies will be described below: With the development of computer technology, the application of multimedia is becoming more and more extensive, and various videos are emerging on the network continuously. When users watch videos, they often miss high-heat segments, high-energy segments or wonderful segments in the videos due to various reasons (such as distraction, multi-tasking, etc.).

[0026] Since these segments are usually the essence of the video content, when users miss watching these segments, it will significantly affect the users' viewing experience and satisfaction.

[0027] Existing video playback platforms mainly rely on methods such as video progress bars and chapter divisions to help users understand target video segments, but none of these methods can accurately remind users of the upcoming target video segments.

[0028] For this reason, the embodiments of the present application provide a video reminder technical solution. In this technical solution, during the video playback process, the current playback time point is monitored in real time, and when the current playback time point is about to reach the start time point of the target video segment (high - popularity segment, wonderful video segment), a user image is collected. When a face image is included in the collected user image, it indicates that the user has not left. At this time, the first reminder method (such as, the video playback frame flashes, the mobile phone vibrates, etc.) is used to perform a reminder operation on the user to remind the user that the target video segment is about to be played, so as to prevent the user from missing the target video segment. When a face image is not included in the collected user image, at this time, the second reminder method (such as, voice broadcast, beeping sound, etc.) is used to perform a reminder operation on the user to remind the user that the target video segment is about to be played, so as to prevent the user from missing the target video segment. In this embodiment, by using different reminder methods based on different scenarios to perform reminder operations on the user, it can be ensured that the user can receive reminders under any circumstances, preventing the user from missing the essence of the video and improving the user's viewing experience and satisfaction. See the following text for details.

[0029] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0030] As Figure 1 shown, the environmental schematic diagram includes a service platform 2, a network 4, and a client 6, where: The service platform 2 can be composed of a single or multiple computing devices. These multiple computing devices can include virtualized computing instances. The virtualized computing instances can include virtual machines, such as emulations of computer systems, operating systems, servers, etc. The computing device can load a virtual machine based on a virtual image and / or other data that defines a specific software (such as an operating system, a dedicated application, a server) for emulation. As the demand for different types of processing services changes, different virtual machines can be loaded and / or terminated on one or more computing devices. A hypervisor can be implemented to manage the use of different virtual machines on the same computing device.

[0031] The service platform 2 can be configured to communicate with the client 6 etc. via the network 4. The network 4 includes various network devices such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network 4 can include physical links such as coaxial cable links, twisted pair cable links, fiber optic links, combinations thereof, etc., or wireless links such as cellular links, satellite links, Wi-Fi links, etc.

[0032] The service platform 2 can provide services such as storage, reading, writing, querying, deleting, etc., such as providing a video reminder service for the client.

[0033] The client 6 can be an electronic device running an operating system such as Windows, Android™, or iOS, such as a smart phone, tablet device, laptop computer, virtual reality device, gaming device, set-top box, in-vehicle terminal, smart TV. Based on the above operating systems, various application programs can be run, such as a video reminder program.

[0034] The client 6 can provide / configure a user access page for manipulating the service platform 2 or uploading an object, etc.

[0035] It should be noted that the above devices are exemplary, and in different scenarios or according to different requirements, the number and types of devices are adjustable.

[0036] The technical solutions of the present application will be introduced below through multiple embodiments. It should be noted that these embodiments can be implemented in various different forms and should not be construed as being limited only to the embodiments described herein.

[0037] Embodiment 1 Figure 2 A flowchart of the video reminder method according to Embodiment 1 of the present application is schematically shown.

[0038] As Figure 2 shown, the video reminder method can include steps S200 to S206, where: Step S200, during the video playback process, continuously monitor the current playback time point.

[0039] Step S202, when the distance between the current playback time point and the start time point of the target video segment is a preset duration, collect a user image.

[0040] Step S204, perform face recognition on the user image, and when it is recognized that the user image contains a face image, perform a reminder operation on the user using a first reminder method.

[0041] Step S206, when it is recognized that the user image does not contain a face image, a second reminder method is used to remind the user.

[0042] The video reminder method provided in this embodiment monitors the current playback time point in real time during the video playback process, and captures a user image when the current playback time point is about to reach the start time point of the target video segment (high - popularity segment, wonderful video segment). When the captured user image contains a face image, it indicates that the user has not left. At this time, a first reminder method (such as the video playback frame flashing, the mobile phone vibrating, etc.) is used to remind the user that the target video segment is about to be played, so as to prevent the user from missing the target video segment. When the captured user image does not contain a face image, a second reminder method (such as voice broadcast, beeping sound, etc.) is used to remind the user that the target video segment is about to be played, so as to prevent the user from missing the target video segment. In this embodiment, by using different reminder methods based on different scenarios to remind the user, the user can receive reminders in any case, avoiding the user missing the essence of the video and improving the user's viewing experience and satisfaction.

[0043] The following combines Figure 1 to elaborate in detail on each step in steps S200 - S206 and other optional steps.

[0044] Step S200 During the video playback process, the current playback time point is monitored in real time.

[0045] The video duration of the video is not limited, and the video type is not limited either. For example, the video type of the video can include sports events, TV dramas, movies, etc.

[0046] In this embodiment, by monitoring the current playback time point in real time during the video playback process, it can be detected in time when the video is about to play to the start time point of the target video segment.

[0047] Step S202 When the distance between the current playback time point and the start time point of the target video segment is a preset duration, a user image is captured.

[0048] The preset duration can be set according to the actual situation. For example, the preset duration is 5 seconds.

[0049] The target video segment can be a high - popularity segment or a wonderful video segment in the video.

[0050] Among them, high-profile clips refer to the parts of the video that can attract a large number of viewers to pay attention, discuss, comment and spread. These clips are often highly popular and topical, and can trigger widespread dissemination and interaction on social media, video platforms and other channels. For example, a classic line or a touching scene in a popular movie may become a high-profile clip due to widespread discussion and sharing by the audience.

[0051] Wonderful video clips is a relatively broad concept, referring to the parts of a video that have high viewing value (for example, strong visual impact, exciting plots or exciting content), artistic value, or entertainment value. These clips may include wonderful performances, clever plot arrangements, beautiful pictures, etc., which can leave a deep impression on the audience. For example, in a concert, the performer's highly skilled solo part, or in a documentary, the picture showing rare natural phenomena, can be regarded as wonderful video clips. For example, in an action movie, the fighting scene where the protagonist counterattacks and kills the opponent in desperation, or in a suspense film, the key moment when the protagonist is about to uncover the mystery, may be considered a wonderful video clip.

[0052] In some implementations, when the terminal device has a built-in camera (camera), the built-in camera of the terminal device can be used to collect user images. When the terminal device does not have a built-in camera (camera), an external camera can be used to collect user images.

[0053] As an example, the starting time point of the target video clip is 30 minutes and 50 seconds, and the preset duration is 10 seconds. When it is detected that the current playback time point is 30 minutes and 40 seconds, the camera device will be scheduled to capture the user image.

[0054] In this embodiment, the user image is collected when the video is about to play to the target video segment, so that the reminder method can be determined based on the collected user image to perform a reminder operation on the user.

[0055] Step S204 , performing face recognition on the user image, and when it is recognized that the user image contains a face image, using a first reminder method to perform a reminder operation on the user.

[0056] In some implementations, after the user image is collected, a face recognition algorithm may be used to perform face recognition on the user image to determine whether a face image exists in the collected user image.

[0057] In one embodiment, after performing face recognition on an image, the recognition result may be that the user image contains one face image, multiple face images, or no face image. When the recognition result is multiple face images, the face image closest to the center point among the multiple face images may be used as the face image for subsequent detection and calculation.

[0058] Among them, the first reminder method may include the video playback frame flashing, the terminal device vibrating, the terminal device emitting a beeping sound, the terminal device screen changing color, the terminal device emitting voice playback, etc.

[0059] In this embodiment, when it is recognized that the user image contains a face image, it indicates that the user has probably not left. At this time, the first reminder method will be used to remind the user to remind the user that the target video segment is about to be played, so as to prevent the user from missing the essence of the video.

[0060] In an alternative embodiment, the first reminder method includes reminder means and reminder amplitude.

[0061] Among them, the reminder means refers to different methods and channels used to remind the user, and the reminder means includes visual reminder, auditory reminder, tactile reminder, etc.

[0062] Visual reminder means reminding the user by displaying text, icons or animations on the screen. For example, when playing a video, the flashing at the edge of the screen or the popped-up text prompt.

[0063] Auditory reminder means reminding the user by sound or voice prompt. For example, playing a piece of reminder sound or voice message.

[0064] Tactile reminder means reminding the user by the vibration or physical feedback of the terminal device. For example, the vibration of a mobile phone or a tablet computer.

[0065] Among them, the reminder amplitude refers to the intensity or degree of reminder, which can be achieved by adjusting the parameters of different reminder means.

[0066] The amplitude of visual reminder can be controlled by adjusting the flashing frequency, the size of the text, the vividness of the color, etc.

[0067] The amplitude of auditory reminder can be controlled by adjusting the volume, pitch or duration of the sound.

[0068] The amplitude of tactile reminder can be controlled by adjusting the intensity and duration of the vibration.

[0069] Refer to Figure 3 , step S204 may include: Step S300, when it is recognized that the user image contains a face image, obtain the key feature points of the face.

[0070] Step S302: Determine whether the user's line of sight is on the screen where the video is played based on the key feature points.

[0071] Step S304: When the user's line of sight is on the screen, determine the first reminder means and the first reminder amplitude for reminding the user.

[0072] Step S306: Remind the user by using the determined first reminder means and the first reminder amplitude.

[0073] The key feature points of the human face may include the key points of the face contour, eyes, eyebrows, nose, mouth and other key parts. For example, the key feature points of the eye part may be the center points of the left and right eyes, the inner corner points of the left and right eyes, the outer corner points of the left and right eyes, etc. The key feature points of the eyebrow part may be the peak points of the left and right eyebrows, the starting points of the left and right eyebrows, etc. The key feature points of the nose part may be the position point of the nose tip, the position points of the left and right nostrils, etc. The key feature points of the mouth part may be the corner points of the left and right lips, the center point of the mouth, etc.

[0074] After obtaining the key feature points of the human face, based on the coordinates of the obtained key feature points, the line of sight of the user can be determined whether it is on the screen where the video is played by means of geometric calculation.

[0075] In one embodiment, the pitch angle can be calculated by analyzing the vertical distance between the center points of the two eyes and the nose tip, and then based on the calculated pitch angle, it can be determined whether the user's line of sight is on the screen where the video is played.

[0076] In another embodiment, the horizontal distance between the nose tip and the left and right corners of the mouth can also be analyzed to calculate the left and right yaw angles, and then, based on the calculated left and right yaw angles, it can be determined whether the user's line of sight is on the screen where the video is played.

[0077] In other embodiments, the line of sight of the user can also be determined whether it is on the screen where the video is played by calculating the distance between other key feature points.

[0078] In this embodiment, when it is determined that the user's line of sight is on the screen, it will be further determined what reminder means and reminder amplitude need to be used to remind the user.

[0079] In one embodiment, when it is determined that the user's line of sight is on the screen, the default reminder means and the default reminder amplitude can be used as the first reminder means and the first reminder amplitude for reminding the user respectively. After that, the determined first reminder means and the first reminder amplitude will be used to remind the user to avoid missing the essence of the video.

[0080] In another embodiment, when it is determined that the user's line of sight is on the screen, the first reminder means and the first reminder amplitude for reminding the user can also be determined according to the user's line of sight angle.

[0081] In this embodiment, when it is recognized that the user image includes a face image, the reminder means and the reminder amplitude for reminding the user will be further determined based on whether the user's video is on the screen, so as to achieve more flexible reminder control and enhance the user experience.

[0082] In an alternative embodiment, the method further includes: When the user's line of sight is outside the screen, determine the second reminder means and the second reminder amplitude for reminding the user; use the determined second reminder means and the second reminder amplitude to perform a reminder operation on the user.

[0083] In some embodiments, when the user's line of sight is outside the screen, in order to better remind the user, the user can be reminded by the second reminder means and the second reminder amplitude.

[0084] Among them, the second reminder means can be in the form of voice or vibration. The intensity of the second reminder amplitude is stronger than that of the first reminder amplitude.

[0085] In this embodiment, when the user's line of sight is outside the screen, the user is reminded by the second reminder means and the second reminder amplitude, that is, a more suitable reminder means and reminder amplitude are used, so as to effectively prevent the user from missing the essence of the video.

[0086] In an alternative embodiment, referring to Figure 4 , determining whether the user's line of sight is on the screen for playing the video based on the key feature points includes: Step S400, determining the user's line of sight angle based on the key feature points.

[0087] Step S402, determining whether the user's line of sight is on the screen for playing the video based on the line of sight angle.

[0088] In one embodiment, the pitch angle can be calculated by analyzing the vertical distance between the center point of the two eyes and the tip of the nose, and then the user's line of sight angle can be determined based on the pitch angle.

[0089] In another embodiment, the horizontal distance between the tip of the nose and the left and right corners of the mouth can also be analyzed to calculate the left and right yaw angles, and then the user's line of sight angle can be determined based on the left and right yaw angles.

[0090] After determining the user's line-of-sight angle, it can be further determined whether the line-of-sight angle is within a preset angle range. If it is within the preset angle range, it can be determined that the user's line of sight is on the screen for playing the video. If it is outside the preset angle range, it can be determined that the user's line of sight is outside the screen for playing the video. Among them, the preset angle range can be set according to the actual situation, and its specific range value is not limited in this embodiment.

[0091] In this embodiment, since the line-of-sight angle can reflect the orientation of the user's head, based on this line-of-sight angle, it is convenient and accurate to determine whether the user's line of sight is on the screen for playing the video.

[0092] In an alternative embodiment, the key feature points include the center points of the left eye, the center points of the right eye, the position point of the nose tip, and the center point of the mouth. Refer to Figure 5 , determining whether the user's line of sight is on the screen for playing the video based on the key feature points includes: Step S500, connecting the center point of the left eye, the center point of the right eye, and the center point of the mouth to form a triangle.

[0093] Step S502, determining whether the position of the nose tip is within the triangle.

[0094] Step S504, when the position of the nose tip is within the triangle, calculating the first distance between the center point of the left eye and the center point of the right eye.

[0095] Step S506, obtaining the target center point of the line connecting the center point of the left eye and the center point of the right eye.

[0096] Step S508, calculating the second distance between the target center point and the center point of the mouth.

[0097] Step S510, calculating the ratio of the first distance to the second distance.

[0098] Step S512, when the ratio is within a preset range, determining that the user's line of sight is on the screen.

[0099] In some embodiments, refer to Figure 6 、 Figure 7After obtaining the left eye center point E1, right eye center point E2, the position point P1 of the nose tip, and the mouth center point M1 according to the face recognition algorithm, a triangle T1 can be formed by connecting the left eye center point E1, the right eye center point E2, and the mouth center point M1, and then it will be calculated whether the position P1 of the nose tip is inside the triangle T1. If the position P1 of the nose tip is inside the triangle T1, the first distance D1 between the left eye center point E1 and the right eye center point E2 will be further calculated, and the target center point P2 of the line connecting the left eye center point E1 and the right eye center point E2 will be obtained. After that, the second distance D2 between the target center point P2 and the mouth center point M1 will be calculated. Finally, the ratio of the first distance D1 to the second distance D2 will be calculated: D1 / D2. When D1 / D2 is within the preset range, it indicates that the user's line of sight angle is small. At this time, it can be determined that the user's line of sight is on the screen.

[0100] Among them, the preset range is set in advance, and its range value can be 0.7 - 0.95.

[0101] In this embodiment, through the above algorithm, it can be more accurately determined whether the user's line of sight is on the screen.

[0102] In an alternative embodiment, the determining whether the user's line of sight is on the screen for playing the video based on the key feature points further includes: When the position of the nose tip is outside the triangle, it is determined that the user's line of sight is outside the screen; or When the ratio is outside the preset range, it is determined that the user's line of sight is outside the screen.

[0103] In this embodiment, when the position of the nose tip is outside the triangle, it indicates that the user's line of sight angle is large. At this time, it can be directly determined that the user's line of sight is outside the screen. When the ratio is outside the preset range, it also indicates that the user's line of sight angle is large. At this time, it can be directly determined that the user's line of sight is outside the screen.

[0104] In an alternative embodiment, referring to Figure 8 , the determining the first reminder means and the first reminder amplitude for reminding the user when the user's line of sight is on the screen includes: Step S800, when the user's line of sight is on the screen, determining the user's line of sight angle based on the key feature points.

[0105] Step S802, determining the first reminder amplitude based on the line of sight angle.

[0106] Step S804, determining the first reminder means based on the video playback platform.

[0107] In some embodiments, a correspondence between different line-of-sight angles and different first reminder amplitudes can be established in advance, and a correspondence between different playback platforms and different first reminder means can be established. For example, when the line-of-sight angles are 10 - 20 degrees, 20 - 30 degrees, and 30 - 40 degrees respectively, the corresponding first reminder amplitudes are slight amplitude reminder, medium amplitude reminder, strong amplitude reminder, and extra strong amplitude reminder. Also, for example, when the playback platform of the video is a mobile phone APP, the corresponding first reminder means is to give a reminder by vibration. When the playback platform of the video is a mobile phone web page, the corresponding first reminder means is to give a reminder by a specific prompt sound or a pop-up window. When the playback platform of the video is a computer APP, the corresponding first reminder means is to give a reminder by the flashing of the video frame. When the playback platform of the video is a computer web page, the corresponding first reminder means is to give a reminder by a beeping sound.

[0108] It should be noted that the above-mentioned slight amplitude reminder, medium amplitude reminder, strong amplitude reminder, and extra strong amplitude reminder are achieved by setting the parameters of the corresponding reminder means.

[0109] In this embodiment, when the user's line of sight is on the screen, the line-of-sight angle of the user is determined based on the key feature points, and after determining the line-of-sight angle, the first reminder amplitude can be determined based on the line-of-sight angle, so as to dynamically adjust the strength of the reminder. For example, when the line of sight deviates from the screen more, the reminder amplitude is increased (such as the slight vibration of the mobile phone app is strengthened, the flashing of the video frame of the computer app is accelerated, etc.); when the line of sight deviates less, the reminder amplitude is weakened. In addition, this embodiment can also flexibly select the corresponding reminder means based on the playback platform of the video, thereby improving the user experience.

[0110] Step S206 , in the case where the user image does not contain a face image, a second reminder method is used to perform a reminder operation on the user.

[0111] When the user image collected does not contain a face image, at this time, a second reminder method (such as voice broadcast, beeping sound, etc.) is used to perform a reminder operation on the user to remind the user that the target video segment is about to be played, so as to prevent the user from missing the viewing of the target video segment.

[0112] In this embodiment, by using different reminder methods based on different scenarios to perform reminder operations on the user, the user can receive reminders in any case, preventing the user from missing the target video segment and improving the user's viewing experience and satisfaction.

[0113] In an alternative embodiment, the method further includes: Determine the target video segment in the video.

[0114] In one embodiment, the target video segment can be pre-marked by the video author.

[0115] In another embodiment, it is also possible to count the positions of the top N in terms of the number of barrage displays, and then use the video segments corresponding to these positions as the target video segments.

[0116] In another embodiment, it is also possible to use the first N segments in the segments that the user drags the progress bar to play repeatedly as the target video segments.

[0117] It should be noted that the methods for determining the target video segment in the video are not limited to the above several. In other embodiments, it is also possible to analyze the scene changes and audio features of the video to identify the wonderful video segments in the video as the target video segments. In other embodiments, it is also possible to analyze the video through a deep learning model to determine the target video segment therefrom.

[0118] In this embodiment, by pre-determining the target video segment in the video, it is convenient to perform a reminder operation on the user based on the target video segment subsequently.

[0119] Embodiment 2 Figure 9 Schematically shows a block diagram of a video reminder device 900 according to Embodiment 2 of the present application. The device can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. The program modules referred to in the embodiments of the present application refer to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. As Figure 9 shown, the device 900 may include: a monitoring module 910, a collection module 920, a first reminder module 930, and a second reminder module 940, where: The monitoring module 910 is configured to monitor the current playback time point in real time during video playback; The collection module 920 is configured to collect a user image when the distance between the current playback time point and the start time point of the target video segment is a preset duration; The first reminder module 930 is configured to perform face recognition on the user image, and when it is recognized that the user image contains a face image, perform a reminder operation on the user in a first reminder manner; The second reminder module 940 is configured to perform a reminder operation on the user in a second reminder manner when it is recognized that the user image does not contain a face image.

[0120] As an alternative embodiment, the first reminder method includes a reminder means and a reminder amplitude. When it is recognized that the user image contains a face image, a reminder operation is performed on the user using the first reminder method, including: When it is recognized that the user image contains a face image, obtain the key feature points of the face; Based on the key feature points, determine whether the user's line of sight is on the screen where the video is played; When the user's line of sight is on the screen, determine the first reminder means and the first reminder amplitude for performing a reminder operation on the user; Perform a reminder operation on the user using the determined first reminder means and the first reminder amplitude.

[0121] As an alternative embodiment, the apparatus 900 is further configured to: When the user's line of sight is outside the screen, determine the second reminder means and the second reminder amplitude for performing a reminder operation on the user; Perform a reminder operation on the user using the determined second reminder means and the second reminder amplitude.

[0122] As an alternative embodiment, the determining whether the user's line of sight is on the screen where the video is played based on the key feature points includes: Based on the key feature points, determine the user's line of sight angle; Based on the line of sight angle, determine whether the user's line of sight is on the screen where the video is played.

[0123] As an alternative embodiment, the key feature points include the center point of the left eye, the center point of the right eye, the position point of the nose tip, and the center point of the mouth. The determining whether the user's line of sight is on the screen where the video is played based on the key feature points includes: Connect the center point of the left eye, the center point of the right eye, and the center point of the mouth to form a triangle; Determine whether the position of the nose tip is within the triangle; When the position of the nose tip is within the triangle, calculate the first distance between the center point of the left eye and the center point of the right eye; Obtain the target center point of the line connecting the center point of the left eye and the center point of the right eye; Calculate the second distance between the target center point and the center point of the mouth; Calculate the ratio of the first distance to the second distance; When the ratio is within a preset range, determine that the user's line of sight is on the screen.

[0124] As an alternative embodiment, determining whether the user's line of sight is on the screen for playing the video based on the key feature points further includes: When the position of the tip of the nose is outside the triangle, determining that the user's line of sight is outside the screen; or When the ratio is outside the preset range, determining that the user's line of sight is outside the screen.

[0125] As an alternative embodiment, when the user's line of sight is on the screen, determining the first reminder means and the first reminder amplitude for reminding the user includes: When the user's line of sight is on the screen, determining the user's line of sight angle based on the key feature points; Determining the first reminder amplitude based on the line of sight angle; Determining the first reminder means based on the video playback platform.

[0126] As an alternative embodiment, the apparatus 900 is further configured to: Determine the target video segment in the video.

[0127] Embodiment III Figure 10 Schematically shows a hardware architecture diagram of a computer device 10000 suitable for implementing the video reminder method according to Embodiment III of the present application. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. As Figure 10 shown, the computer device 10000 includes, but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium. The readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disc, etc. In some embodiments, the memory 10010 may be an internal storage module of the computer device 10000, such as the hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. equipped on the computer device 10000. Of course, the memory 10010 may also include both the internal storage module and the external storage device of the computer device 10000. In this embodiment, the memory 10010 is generally used to store the operating system and various application software installed on the computer device 10000, such as the program code of the video reminder method. In addition, the memory 10010 may also be used to temporarily store various types of data that have been output or will be output.

[0128] In some embodiments, the processor 10020 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other chip. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0129] The network interface 10030 may include a wireless network interface or a wired network interface, which is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal via a network, and establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an enterprise intranet (Intranet), the Internet, the Global System of Mobile communication (GSM for short), Wideband Code Division Multiple Access (WCDMA for short), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0130] It should be noted that Figure 10 Only the computer device with components 10010 - 10030 is shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0131] In this embodiment, the video reminder method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as the processor 10020) to complete the embodiments of the present application.

[0132] Embodiment 4 The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the video reminder method in the embodiments are implemented.

[0133] In this embodiment, the computer-readable storage medium includes flash memory, hard disks, multimedia cards, card-type memories (such as SD or DX memories, etc.), random access memories (RAM), static random access memories (SRAM), read-only memories (ROM), electrically erasable programmable read-only memories (EEPROM), programmable read-only memories (PROM), magnetic memories, magnetic disks, optical discs, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc., equipped on the computer device. Of course, the computer-readable storage medium may also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer-readable storage medium is generally used to store the operating system installed on the computer device and various application software, such as the program code of the video reminder method in the embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various data that have been output or will be output.

[0134] Embodiment 5 The embodiment of the present application also provides a computer program product, including a computer program, which when executed by a processor implements the method in the above embodiment.

[0135] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the embodiments of the present application can be implemented by a general computer device. They can be concentrated on a single computer device or distributed on a network composed of multiple computer devices. Optionally, they can be implemented by program codes executable by the computer device, so that they can be stored in a storage device and executed by the computer device. And in some cases, the steps shown or described can be executed in a different order from here, or they can be separately made into individual integrated circuit modules, or multiple modules or steps among them can be made into a single integrated circuit module to implement. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0136] It should be noted that the above are only the preferred embodiments of the present application, and do not limit the patent protection scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A video reminder method, characterized in that: The method comprises: During video playback, the current playback time point is monitored in real time; When the distance between the current playback time point and the start time point of the target video segment is a preset time length, capturing a user image; Performing face recognition on the user image, and when it is recognized that the user image contains a face image, using a first reminder method to perform a reminder operation on the user; When it is identified that the user image does not contain a face image, a second reminder method is used to remind the user.

2. The method according to claim 1, characterized in that The first reminder method includes a reminder means and a reminder amplitude. When it is recognized that the user image contains a face image, the first reminder method is used to remind the user, including: When it is recognized that the user image contains a face image, obtaining key feature points of the face; Determining whether the user's sight line is located on the screen playing the video based on the key feature points; When the user's sight is located on the screen, determining a first reminder means and a first reminder amplitude for performing a reminder operation on the user; The determined first reminder means and first reminder amplitude are used to perform a reminder operation on the user.

3. The method according to claim 2, characterized in that The method further comprises: When the user's line of sight is outside the screen, determining a second reminder means and a second reminder amplitude for performing a reminder operation on the user; The determined second reminder means and second reminder amplitude are used to remind the user.

4. The method according to claim 2, characterized in that: The determining, based on the key feature points, whether the user's sight line is located on the screen playing the video includes: Determining the user's sight angle based on the key feature points; Based on the sight line angle, it is determined whether the user's sight line is located on the screen playing the video.

5. The method according to claim 2, characterized in that: The key feature points include a left eye center point, a right eye center point, a nose tip position point, and a mouth center point. Determining whether the user's sight line is located on the screen playing the video based on the key feature points includes: Connect the center point of the left eye, the center point of the right eye and the center point of the mouth to form a triangle; determining whether the position of the nose tip is within the triangle; When the position of the nose tip is within the triangle, calculating a first distance between the center point of the left eye and the center point of the right eye; Obtain a target center point of a line connecting the left eye center point and the right eye center point; Calculating a second distance between the target center point and the mouth center point; calculating a ratio of the first distance to the second distance; When the ratio is within a preset range, it is determined that the user's sight line is located on the screen.

6. The method according to claim 5, characterized in that The determining, based on the key feature points, whether the user's sight line is located on the screen playing the video further includes: When the position of the nose tip is outside the triangle, determining that the user's line of sight is outside the screen; or When the ratio is outside the preset range, it is determined that the user's line of sight is outside the screen.

7. The method according to any one of claims 2 to 5, characterized in that: The step of determining a first reminder means and a first reminder amplitude for performing a reminder operation on the user when the user's sight line is located on the screen includes: When the user's sight line is located on the screen, determining the user's sight line angle based on the key feature point; determining the first reminder amplitude based on the sight angle; The first reminder means is determined based on a playback platform of the video.

8. The method according to claim 1, characterized in that The method further comprises: A target video segment in the video is determined.

9. A video reminder device, characterized in that: The device comprises: The monitoring module is used to monitor the current playback time point in real time during video playback; A collection module, used for collecting a user image when the distance between the current playback time point and the start time point of the target video segment is a preset time length; A first reminder module, configured to perform face recognition on the user image, and when it is recognized that the user image contains a face image, use a first reminder method to perform a reminder operation on the user; The second reminder module is used to use a second reminder method to remind the user when it is recognized that the user image does not contain a face image.

10. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 8 are implemented.

Images