Intelligent video playback device

Abstract

The present invention relates to an intelligent video playback device, which belongs to the field of smart home technology and solves the problem that the existing video playback device cannot automatically adapt to various user postures and intelligently control the playback state. The device includes: a face recognition module, which is used to recognize the position and direction of the face in the collected user face image, and send a posture adjustment instruction to a mobile video playback device according to the recognition result; a voice recognition module, which is used to receive the user's real-time voice information, recognize the video control instruction contained in the information, and send the video control instruction to the mobile video playback device; the mobile video playback device is used to control the display to move to the best playback angle facing the face according to the obtained posture adjustment instruction, and control the playback state of the current video according to the video control instruction.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a smart video playback device. Background Art

[0002] With the development of science and technology, people have put forward higher requirements for their living environment. Smart home, as a product of the development of the Internet and artificial intelligence, has become popular. Realizing the automation and intelligence of home appliance control has become a way and trend for people to pursue a comfortable, convenient and safe home life. Compared with traditional video playback methods, the current smart video playback device not only has richer playback content, but also provides interactive options to improve user entertainment.

[0003] At present, there are two main types of smart video playback devices: fixed installation type and handheld type. For fixed installation video playback devices, users must watch in a fixed position. For handheld video playback devices, users need to hold or use tools to support them in a fixed position. Both of the above video playback devices cannot adapt well to the viewer's posture. Watching videos for a long time in a fixed posture or handheld mode will cause physical fatigue to the viewer, and even cause cervical, lumbar, and wrist diseases.

[0004] In addition, when people are watching a video playback device, they are often reluctant to shift their eyes away from the video screen, and often need to perform some remote control operations during the viewing process. The existing technology is to achieve remote control operations by pressing physical buttons on the remote control device, which affects the smoothness of the user's video viewing. Currently, there is a lack of an intelligent video playback device that can automatically adapt to various user postures and intelligently control the playback state. Summary of the invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide an intelligent video playback device to solve the problem that the existing video playback device cannot automatically adapt to various user postures and intelligently control the playback state.

[0006] In one aspect, an embodiment of the present invention provides a video playback device, comprising:

[0007] A face recognition module is used to identify the position and direction of the face in the collected user face image, and issue a posture adjustment instruction to the mobile video playback device according to the recognition result;

[0008] A voice recognition module, used to receive real-time voice information from a user, recognize video control instructions contained in the information, and send the video control instructions to a mobile video playback device;

[0009] The mobile video playback device is used to control the display to move to the best playback viewing angle facing the face of the person according to the obtained posture adjustment instruction, and control the playback state of the current video according to the video control instruction.

[0010] The beneficial effects of the above technical solution are as follows: through the automatic collection and processing of user postures and audio signals, the intelligent position control and intelligent playback state of the video playback device are realized, which greatly improves the user's comfort.

[0011] Based on further improvements of the above-mentioned device, the voice recognition module is also used to identify posture adjustment instruction 2 contained in the current real-time voice information and send it to the mobile video playback device after the mobile video playback device controls the display to move to the optimal playback viewing angle facing the face according to posture adjustment instruction 1; wherein, posture adjustment instruction 2 is used to control the display of the mobile video playback device to perform corresponding movement or rotation operations.

[0012] The beneficial effects of the above further improvement scheme are: by improving the function of the voice recognition module, the response range of the device to voice commands is enriched, which can effectively compensate for the collection errors or failures of the face recognition module and better meet the actual needs of users.

[0013] Furthermore, the intelligent video playback device also includes:

[0014] The sleep sensing module is used to send a sleep sensing instruction to the face recognition module at regular intervals after the mobile video playback device is started, so as to control the face recognition module to collect the user's eye information within a preset time. After receiving the eye information, the module analyzes and determines whether the user has entered a sleep state. If the user has entered a sleep state, a shutdown control instruction is sent to control the mobile video playback device to automatically shut down. Otherwise, the video playback state of the mobile video playback device is maintained.

[0015] The beneficial effect of the above further improvement scheme is: the device can automatically shut down after the user falls asleep, which effectively reduces the power consumption of the smart video playback device and reduces the radiation of video light to the face after the user falls asleep, which can effectively improve the user's sleep quality.

[0016] Furthermore, the face recognition module further comprises:

[0017] High-definition camera, used to collect user face images and send them to the pre-processing submodule;

[0018] The preprocessing submodule is used to filter, identify contours, enhance images, and extract features of the received user face image in sequence, and send the acquired feature information of each part of the face to the data processing submodule;

[0019] The data processing submodule is used to obtain the current position and direction of the face based on the received feature information of each part of the face in combination with the built-in camera coordinate system, and then send a posture adjustment instruction containing the current position and direction of the face to the mobile video playback device.

[0020] The beneficial effect of the above further improvement scheme is: the effective collection and processing of three-dimensional facial information is realized through the face recognition module, which lays a solid foundation for the intelligent synchronization of mobile video playback equipment and facial orientation.

[0021] Furthermore, the preprocessing submodule executes the following procedure to obtain feature information of each part of the face:

[0022] Detecting the face area in the user's face image;

[0023] Enlarging the face region, and performing Gaussian high-pass filtering on the image details in the region to obtain an edge-enhanced face image;

[0024] Performing contour recognition on the edge-enhanced face image to obtain four edge lines on the face: upper, lower, left, and right;

[0025] Perform NMF decomposition on the area formed by the above four edge lines to obtain the shapes of the user's eyes, nose, mouth, ears and the position information of each point;

[0026] The obtained shapes of the user's eyes, nose, mouth, and ears, as well as the position information of each point, are sent to the data processing submodule as feature information of each part of the face.

[0027] The beneficial effect of the above further improvement scheme is that the accuracy and effectiveness of the collected face information are improved through the above image preprocessing of the preprocessing submodule, thereby laying a solid foundation for the accuracy of automatic posture adjustment.

[0028] Furthermore, the data processing submodule executes the following procedure to obtain the current position and direction of the face:

[0029] Receive the shape and position information of the user's eyes, nose, mouth, and ears;

[0030] Determine the current direction of the face according to the shape of the user's eyes or ears, wherein the current direction of the face is toward the side with smaller eyes or ears;

[0031] Determine the horizontal and vertical central axes of the user's face according to the shape of the user's nose and mouth, and further determine the pitch angle and yaw angle of the face in the camera coordinate system, wherein the pitch angle is the angle between the longitudinal central axis of the face and the positive direction of the y-axis, and the yaw angle is the angle between the horizontal central axis and the positive direction of the x-axis; the origin of the camera coordinate system is the camera layout position, the negative direction of the y-axis is the shooting direction, and the negative direction of the z-axis is vertically downward;

[0032] According to the obtained pitch angle and yaw angle, the x-, y-, and z-direction coordinates of the face position are determined in combination with the position coordinates of the user's eyes, nose, mouth, and ears; wherein the x-direction coordinate X of the face position is obtained by the following formula:

[0033]

[0034] Where x1 is the x-coordinate of the center of the line connecting the user's two eyes, x2 is the x-coordinate of the tip of the user's nose, x3 is the x-coordinate of the center of the mouth, and x4 is the x-coordinate of the center of the line connecting the user's two ears;

[0035] The y-coordinate Y of the face position is obtained by the following formula:

[0036]

[0037] Where a is the deflection angle of the face, l1 is the length of the line between the user's two eyes, l2 is the length of the line between the user's two ears, and k1 and k2 are pre-calibrated proportional coefficients;

[0038] The z-coordinate Z of the face position is obtained by the following formula:

[0039]

[0040] Wherein, z1 is the z-coordinate of the center of the line connecting the two eyes of the user, z2 is the z-coordinate of the tip of the nose of the user, z3 is the z-coordinate of the center of the mouth, z4 is the z-coordinate of the center of the line connecting the two ears of the user, and k3~k6 are pre-calibrated proportional coefficients.

[0041] The beneficial effect of the above further improvement scheme is that it can effectively obtain the current user's face orientation and specific angle, including position coordinates, pitch angle, and deflection angle, thereby improving the accuracy of face information recognition.

[0042] Furthermore, the mobile video playback device further comprises:

[0043] The control submodule is used to control the face recognition module and the voice recognition module to start up, receive the posture adjustment instructions 1 and 2 and the video control instruction in real time, control the moving component to drive the display to move to a preset distance from the current position of the face according to the posture adjustment instruction 1, and control the rotating component to drive the display to rotate toward the current direction of the face until the position of the display remains unchanged, thereby completing the automatic posture adjustment, and then control the moving component or the rotating component to drive the display to perform the corresponding movement or rotation operation according to the posture adjustment instruction 2 to complete the manual posture adjustment; after the movement or rotation operation is completed, control the display to display the status in real time according to the video control instruction;

[0044] A moving component, used to drive the display to move according to the control of the control submodule;

[0045] A rotating assembly, used to drive the display to rotate according to the control of the control submodule;

[0046] Display, used to display the current video playback status.

[0047] The beneficial effect of the above further improvement scheme is that: through the hybrid mode of automatic posture adjustment and manual posture adjustment, the intelligent adjustment of the posture of the display of the mobile video playback device is realized, which can better meet the actual needs of users.

[0048] Further, the control submodule executes the following procedures:

[0049] Control the start-up of face recognition module and voice recognition module;

[0050] Receive attitude adjustment instructions 1 and 2 and video control instructions;

[0051] According to the posture adjustment instruction, the moving component is controlled to drive the display to move to a preset distance from the current position of the face, and the rotating component is controlled to drive the display to rotate toward the current direction of the face to complete the automatic posture adjustment;

[0052] After the automatic posture adjustment is completed, the moving component or the rotating component is controlled according to the second posture adjustment instruction to drive the display to perform the corresponding movement or rotation operation to complete the manual posture adjustment;

[0053] After the manual posture adjustment is completed, the real-time display status of the display is controlled according to the video control instructions; the video control instructions include at least one of modifying the video name, increasing the volume, decreasing the volume, the next video, the previous video, pausing, fast forwarding, and slowing down the playback speed.

[0054] The beneficial effect of the above further improvement scheme is: by reasonably controlling the posture adjustment timing, the convenience of use and user comfort are improved, and the actual needs of users can be fully met.

[0055] Furthermore, the control submodule executes the following procedure to complete the automatic posture adjustment:

[0056] Extracting the current position, pitch angle and yaw angle of the face contained in the posture adjustment instruction 1;

[0057] Controlling the moving component to drive the display to move in a preset manner to a preset distance from the current position of the above-mentioned face, and controlling the rotating component to drive the display to rotate toward the pitch angle and the yaw angle until it rotates to the pitch angle and the yaw angle;

[0058] Control the face recognition module to collect the user's face image again, and identify the position, pitch angle and yaw angle of the face in the image;

[0059] Receiving the posture control instruction 1 sent by the face recognition module again, and extracting the current position, pitch angle and yaw angle of the face contained in the posture adjustment instruction 1;

[0060] According to the posture control instruction obtained by re-identification, the moving component or the rotating component is controlled to drive the display to perform the corresponding movement or rotation operation until the position of the display remains unchanged, thereby obtaining the best playback angle facing the face and completing the automatic posture adjustment.

[0061] The beneficial effect of the above further improvement scheme is that the efficiency and effect of posture adjustment are improved through adaptive automatic posture adjustment.

[0062] Furthermore, the sleep sensing module executes the following program to control the mobile video playback device to automatically shut down:

[0063] After being started according to the control instruction of the mobile video playback device, a sleep sensing instruction is periodically sent to the face recognition module, and the face recognition module is controlled according to the sleep sensing instruction to collect the user's eye shape within a preset time;

[0064] Whether the user has entered a sleep state is determined based on N user eye shapes received within a preset time. If the N user eye shapes are all a straight line, it is determined that the user has entered a sleep state, and a shutdown control command is issued to control the mobile video playback device to automatically shut down; otherwise, the video playback state of the mobile video playback device is maintained.

[0065] The beneficial effect of the above further improvement scheme is that the intelligent adjustment of the playing state of the intelligent video player is realized through intelligent sensing of the shape of the human eye. Since the intelligent video player automatically shuts down when the user is asleep, the user only needs to close his eyes for a preset time to realize automatic shutdown when he does not want to watch.

[0066] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0068] Figure 1 This is a schematic diagram of the composition of an intelligent video playback device according to Embodiment 1 of the present invention;

[0069] Figure 2 This is a schematic diagram of the composition of the intelligent video playback device according to embodiment 2 of the present invention. DETAILED DESCRIPTION

[0070] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0071] Example 1

[0072] A specific embodiment of the present invention discloses an intelligent video playback device, such as Figure 1 As shown, it includes a face recognition module, a voice recognition module, and a mobile video playback device. Optionally, the output ends of the face recognition module and the voice recognition module are respectively connected to the input end of the mobile video playback device; or, the face recognition module, the voice recognition module and the mobile video playback device can also transmit data through a wireless network (all three have built-in wireless communication modules).

[0073] The face recognition module is used to identify the position and direction of the face in the collected user face image, and issue a posture adjustment instruction to the mobile video playback device based on the recognition result.

[0074] The voice recognition module is used to receive the user's real-time voice information, recognize the video control instructions contained in the information, and send the video control instructions to the mobile video playback device.

[0075] The mobile video playback device is used to control the display to move to the best playback viewing angle facing the face of the person according to the obtained posture adjustment instruction, and control the playback state of the current video according to the video control instruction.

[0076] During implementation, after the intelligent video playback device is started, it can automatically identify the position and direction of the face through the face recognition module, and then control the display to move to the best playback angle facing the face, and then intelligently control the playback status of the current video according to the user's real-time voice instructions.

[0077] Compared with the prior art, the device provided in this embodiment realizes intelligent position control and intelligent playback status of the video playback device by automatically collecting and processing user postures and audio signals, thereby greatly improving user comfort.

[0078] Example 2

[0079] Based on the optimization of the intelligent video playback device described in Example 1, the intelligent video playback device may further include a sleep sensing module, such as Figure 2Optionally, the output end of the sleep sensing module is connected to the input end of the mobile video player device, and the data end is connected to the data end of the face recognition module; or, the sleep sensing module, the mobile video player device, and the face recognition module can also be built-in wireless communication modules to transmit data to each other through a wireless network.

[0080] The sleep sensing module is used to send a sleep sensing instruction to the face recognition module at a fixed time after the mobile video playback device is started, and control the face recognition module to collect the user's eye information within a preset time. After receiving the eye information, it analyzes and determines whether the user has entered the sleep state. If the user has entered the sleep state, it sends a shutdown control instruction to control the mobile video playback device to automatically shut down. Otherwise, the video playback state of the mobile video playback device is maintained. The eye information includes the shape of the eye and the coordinates of each point.

[0081] Preferably, the voice recognition module is also used to identify the posture adjustment instruction 2 contained in the current real-time voice information and send it to the mobile video playback device after the mobile video playback device controls the display to move to the best playback viewing angle facing the face according to the posture adjustment instruction 1, wherein the posture adjustment instruction 2 is used to control the display of the mobile video playback device to perform a corresponding movement or rotation operation, including a specific movement distance, direction, and rotation angle. In the case of a failure of the face recognition module, the user can directly adjust the display position and posture through the voice recognition module.

[0082] Preferably, the face recognition module further includes a high-definition camera, a preprocessing submodule, and a data processing submodule which are connected in sequence.

[0083] The high-definition camera is used to collect user face images and send them to the pre-processing submodule.

[0084] The preprocessing submodule is used to filter, identify contours, enhance images, and extract features of the received user face image in sequence, and send the acquired feature information of each part of the face to the data processing submodule. The feature information of each part of the face is the shape of each part of the face and the position coordinates of each point.

[0085] The data processing submodule is used to obtain the current position and direction of the face based on the received feature information of each part of the face in combination with the built-in camera coordinate system, and then send a posture adjustment instruction containing the current position and direction of the face to the mobile video playback device.

[0086] Preferably, the preprocessing submodule executes the following procedure to obtain feature information of each part of the face:

[0087] S1. Detecting the face area in the user's face image;

[0088] S2. Enlarging the face region and performing Gaussian high-pass filtering on the image details in the region to obtain an edge-enhanced face image;

[0089] S3. Perform contour recognition on the edge-enhanced face image to obtain four edge lines on the face, namely, top, bottom, left and right;

[0090] S4. performing non-negative matrix factorization (NMF) decomposition on the region formed by the above four edge lines to obtain the shapes of the user's eyes, nose, mouth, and ears and the position information of each point;

[0091] S5. The obtained shapes of the user's eyes, nose, mouth, and ears, as well as the position information of each point, are sent to the data processing submodule as feature information of each part of the face.

[0092] Preferably, the data processing submodule executes the following procedure to obtain the current position and direction of the face:

[0093] S6. Receive the shape and position information of the user's eyes, nose, mouth, and ears sent by the preprocessing submodule;

[0094] S7. Determine the current direction of the face according to the shape of the user's eyes or ears, wherein the face is currently oriented toward the side with smaller eyes or ears;

[0095] S8. Determine the horizontal and vertical axis lines of the user's face according to the shape of the user's nose and mouth, and further determine the pitch angle and yaw angle of the face in the camera coordinate system, wherein the pitch angle is the angle between the longitudinal axis line of the face and the positive direction of the y-axis, and the yaw angle is the angle between the horizontal axis line and the positive direction of the x-axis; the origin o of the camera coordinate system is the camera layout position, the negative direction of the y-axis is the shooting direction, and the negative direction of the z-axis is vertically downward. The face direction is determined by the above-defined pitch angle and yaw angle. Optionally, the face direction can also be determined by using the coordinate system definition commonly used in the art, that is, the pitch angle is the angle between the longitudinal axis line of the face and the xoy plane, and the yaw angle is the angle between the horizontal axis line and the yoz plane;

[0096] S9. Determine the x-, y-, and z-direction coordinates of the face position based on the pitch angle and yaw angle obtained, combined with the coordinates of the positions of the user's eyes, nose, mouth, and ears; wherein the x-direction coordinate X of the face position is obtained by the following formula:

[0097]

[0098] Wherein, x1 is the x-coordinate of the center of the line connecting the two eyes of the user, x2 is the x-coordinate of the tip of the nose of the user, x3 is the x-coordinate of the center of the mouth, and x4 is the x-coordinate of the center of the line connecting the two ears of the user.

[0099] The y-coordinate Y of the face position is obtained by the following formula:

[0100]

[0101] Where a is the deflection angle of the face, l1 is the length of the line connecting the user's two eyes, l2 is the length of the line connecting the user's two ears, and k1 and k2 are pre-calibrated proportional coefficients.

[0102] The z-coordinate Z of the face position is obtained by the following formula:

[0103]

[0104] Wherein, z1 is the z-coordinate of the center of the line connecting the two eyes of the user, z2 is the z-coordinate of the tip of the nose of the user, z3 is the z-coordinate of the center of the mouth, z4 is the z-coordinate of the center of the line connecting the two ears of the user, and k3~k6 are pre-calibrated proportional coefficients.

[0105] Optionally, X, Y, and Z may also be obtained through other coordinates of the user's eyes, nose, mouth, and ears, as well as other coordinate calculation methods, which can be understood by those skilled in the art.

[0106] Preferably, the mobile video playback device further comprises a control submodule, a mobile component, a rotating component, and a display, wherein the input end of the control submodule is connected to the output end of the face recognition module and the voice recognition module, and the output end thereof is respectively connected to the input end of the mobile component, the rotating component, and the display.

[0107] The control submodule is used to control the face recognition module and the voice recognition module to start up, receive posture adjustment instructions 1 and 2 and video control instructions in real time, control the moving component to drive the display to move to a preset distance from the current position of the face according to the posture adjustment instruction 1, and control the rotating component to drive the display to rotate toward the current direction of the face until the position of the display remains unchanged, thereby completing the automatic posture adjustment, and then control the moving component or the rotating component to drive the display to perform corresponding movement or rotation operations according to the posture adjustment instruction 2 to complete the manual posture adjustment; after the movement or rotation operation is completed, control the display to display the status in real time according to the video control instruction.

[0108] The moving component is used to drive the display to move according to the control of the control submodule.

[0109] The rotating component is used to drive the display to rotate according to the control of the control submodule.

[0110] Display, used to display the current video playback status.

[0111] Preferably, the control submodule executes the following procedures to complete the posture control and video state control:

[0112] SS1. Control the start-up of face recognition module and voice recognition module;

[0113] SS2. Receive attitude adjustment instructions 1, 2 and video control instructions;

[0114] SS3. According to the posture adjustment instruction, the moving component drives the display to move to a preset distance from the current position of the face, and controls the rotating component to drive the display to rotate toward the current direction of the face to complete the automatic posture adjustment;

[0115] SS4. After the automatic posture adjustment is completed, the moving component or the rotating component is controlled according to the posture adjustment instruction 2 to drive the display to perform the corresponding movement or rotation operation to complete the manual posture adjustment;

[0116] SS5. After the manual posture adjustment is completed, the real-time display status of the display is controlled according to the video control instructions; the video control instructions include at least one of changing the video name, increasing the volume, decreasing the volume, the next video, the previous video, pausing, fast forwarding, and slowing down the playback speed.

[0117] Preferably, step SS3 further comprises:

[0118] SS31. Extract the current position, pitch angle and yaw angle of the face contained in the posture adjustment instruction;

[0119] SS32. Control the moving component to drive the display to move in a preset manner to a preset distance from the current position of the above-mentioned face, and control the rotating component to drive the display to rotate toward the pitch angle and yaw angle until it rotates to the pitch angle and yaw angle;

[0120] SS33 controls the face recognition module to collect the user's face image again, and identifies the position, pitch angle and deflection angle of the face in the image;

[0121] SS34 receives the posture control instruction issued by the face recognition module again, and extracts the current position, pitch angle and deflection angle of the face contained in the posture adjustment instruction;

[0122] SS35. According to the posture control instruction obtained by re-identification, the moving component or the rotating component is controlled to drive the display to perform the corresponding movement or rotation operation until the position of the display remains unchanged, thereby obtaining the best viewing angle facing the face and completing the automatic posture adjustment.

[0123] Preferably, the sleep sensing module executes the following program to control the mobile video playback device to automatically shut down:

[0124] SSS1. After the mobile video playback device is started according to the control command, a sleep sensing command is issued to the face recognition module at regular intervals, and the face recognition module is controlled according to the sleep sensing command to collect the user's eye shape within a preset time;

[0125] SSS2. Determine whether the user has entered a sleep state based on N user eye shapes received within a preset time. If the N user eye shapes are all a straight line, it is determined that the user has entered a sleep state, and a shutdown control command is issued to control the mobile video playback device to automatically shut down; otherwise, the video playback state of the mobile video playback device is maintained.

[0126] Compared with Example 1, the intelligent video playback device provided in this embodiment can realize that the device can automatically shut down after the user falls asleep, effectively reducing the power consumption of the intelligent video playback device, and reducing the radiation of the video light to the face after the user falls asleep, which can effectively improve the user's sleep quality. At the same time, the accuracy of the collected user face information and the accuracy of the posture adjustment of the mobile video playback device are further improved. In the case of a failure of the face recognition module, the posture of the mobile video playback device can still be adjusted by artificial voice, further improving the utilization of the device and the user's comfort.

[0127] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0128] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. An intelligent video playback device, characterized in that: include: A face recognition module is used to identify the position and direction of the face in the collected user face image, and issue a posture adjustment instruction to the mobile video playback device according to the recognition result; A voice recognition module, used to receive real-time voice information from a user, recognize video control instructions contained in the information, and send the video control instructions to a mobile video playback device; A mobile video playback device, used to control the display to move to an optimal playback viewing angle facing the face of the person according to the obtained posture adjustment instruction, and to control the playback state of the current video according to the video control instruction; The face recognition module further comprises: High-definition camera, used to collect user face images and send them to the pre-processing submodule; The preprocessing submodule is used to filter, identify contours, enhance images, and extract features of the received user face image in sequence, and send the acquired feature information of each part of the face to the data processing submodule; The data processing submodule is used to obtain the current position and direction of the face according to the received feature information of each part of the face in combination with the built-in camera coordinate system, and then send a posture adjustment instruction containing the current position and direction of the face to the mobile video playback device; The data processing submodule executes the following procedure to obtain the current position and direction of the face: Receive the shape and position information of the user's eyes, nose, mouth, and ears; Determine the current direction of the face according to the shape of the user's eyes or ears, wherein the current direction of the face is toward the side with smaller eyes or ears; Determine the horizontal and vertical central axes of the user's face according to the shape of the user's nose and mouth, and further determine the pitch angle and yaw angle of the face in the camera coordinate system, wherein the pitch angle is the angle between the longitudinal central axis of the face and the positive direction of the y-axis, and the yaw angle is the angle between the horizontal central axis and the positive direction of the x-axis; the origin of the camera coordinate system is the camera layout position, the negative direction of the y-axis is the shooting direction, and the negative direction of the z-axis is vertically downward; According to the obtained pitch angle and yaw angle, the x-, y-, and z-direction coordinates of the face position are determined in combination with the position coordinates of the user's eyes, nose, mouth, and ears; wherein the x-direction coordinate X of the face position is obtained by the following formula: Where x1 is the x-coordinate of the center of the line connecting the user's two eyes, x2 is the x-coordinate of the tip of the user's nose, x3 is the x-coordinate of the center of the mouth, and x4 is the x-coordinate of the center of the line connecting the user's two ears. The y-coordinate Y of the face position is obtained by the following formula: Where a is the deflection angle of the face, l1 is the length of the line between the user's two eyes, l2 is the length of the line between the user's two ears, and k1 and k2 are pre-calibrated proportional coefficients; The z-coordinate Z of the face position is obtained by the following formula: Wherein, z1 is the z-coordinate of the center of the line connecting the two eyes of the user, z2 is the z-coordinate of the tip of the nose of the user, z3 is the z-coordinate of the center of the mouth, z4 is the z-coordinate of the center of the line connecting the two ears of the user, and k3~k6 are pre-calibrated proportional coefficients.

2. The intelligent video playback device according to claim 1, characterized in that: The voice recognition module is also used to identify the posture adjustment instruction 2 contained in the current real-time voice information and send it to the mobile video playback device after the mobile video playback device controls the display to move to the optimal playback viewing angle facing the face according to the posture adjustment instruction 1, wherein the posture adjustment instruction 2 is used to control the display of the mobile video playback device to perform corresponding movement or rotation operations.

3. The intelligent video playback device according to claim 1 or 2, characterized in that: Also includes: The sleep sensing module is used to send a sleep sensing instruction to the face recognition module at regular intervals after the mobile video playback device is started, so as to control the face recognition module to collect the user's eye information within a preset time. After receiving the eye information, the module analyzes and determines whether the user has entered a sleep state. If the user has entered a sleep state, a shutdown control instruction is sent to control the mobile video playback device to automatically shut down. Otherwise, the video playback state of the mobile video playback device is maintained.

4. The intelligent video playback device according to claim 3, characterized in that: The preprocessing submodule executes the following procedure to obtain the feature information of each part of the face: Detecting the face area in the user's face image; Enlarging the face region, and performing Gaussian high-pass filtering on the image details in the region to obtain an edge-enhanced face image; Performing contour recognition on the edge-enhanced face image to obtain four edge lines on the face: upper, lower, left, and right; Perform NMF decomposition on the area formed by the above four edge lines to obtain the shapes of the user's eyes, nose, mouth, ears and the position information of each point; The obtained shapes of the user's eyes, nose, mouth, and ears, as well as the position information of each point, are sent to the data processing submodule as feature information of each part of the face.

5. The intelligent video playback device according to claim 2, characterized in that: The mobile video playback device further comprises: The control submodule is used to control the face recognition module and the voice recognition module to start up, receive the posture adjustment instructions 1 and 2 and the video control instruction in real time, control the moving component to drive the display to move to a preset distance from the current position of the face according to the posture adjustment instruction 1, and control the rotating component to drive the display to rotate toward the current direction of the face to complete the automatic posture adjustment, and then control the moving component or the rotating component to drive the display to perform the corresponding movement or rotation operation according to the posture adjustment instruction 2 to complete the manual posture adjustment; after the movement or rotation operation is completed, control the display to display the status in real time according to the video control instruction; A moving component, used to drive the display to move according to the control of the control submodule; A rotating assembly, used to drive the display to rotate according to the control of the control submodule; Display, used to display the current video playback status.

6. The intelligent video playback device according to claim 5, characterized in that: The control submodule executes the following procedures: Control the start-up of face recognition module and voice recognition module; Receive attitude adjustment instructions 1 and 2 and video control instructions; According to the posture adjustment instruction, the moving component is controlled to drive the display to move to a preset distance from the current position of the face, and the rotating component is controlled to drive the display to rotate toward the current direction of the face to complete the automatic posture adjustment; After the automatic posture adjustment is completed, the moving component or the rotating component is controlled according to the second posture adjustment instruction to drive the display to perform the corresponding movement or rotation operation to complete the manual posture adjustment; After the manual posture adjustment is completed, the real-time display status of the display is controlled according to the video control instructions; the video control instructions include at least one of modifying the video name, increasing the volume, decreasing the volume, the next video, the previous video, pausing, fast forwarding, and slowing down the playback speed.

7. The intelligent video playback device according to claim 6, characterized in that: The control submodule executes the following procedure to complete the automatic posture adjustment: Extracting the current position, pitch angle and yaw angle of the face contained in the posture adjustment instruction 1; Controlling the moving component to drive the display to move in a preset manner to a preset distance from the current position of the above-mentioned face, and controlling the rotating component to drive the display to rotate toward the pitch angle and the yaw angle until it rotates to the pitch angle and the yaw angle; Control the face recognition module to collect the user's face image again, and identify the position, pitch angle and yaw angle of the face in the image; Receiving the posture control instruction 1 sent by the face recognition module again, and extracting the current position, pitch angle and yaw angle of the face contained in the posture adjustment instruction 1; According to the posture control instruction obtained by re-identification, the moving component or the rotating component is controlled to drive the display to perform the corresponding movement or rotation operation until the position of the display remains unchanged, thereby obtaining the best playback angle facing the face and completing the automatic posture adjustment.

8. The intelligent video playback device according to claim 3, characterized in that: The sleep sensing module executes the following program to control the mobile video playback device to automatically shut down: After being started according to the control instruction of the mobile video playback device, a sleep sensing instruction is periodically sent to the face recognition module, and the face recognition module is controlled according to the sleep sensing instruction to collect the user's eye shape within a preset time; Whether the user has entered a sleep state is determined based on N user eye shapes received within a preset time. If the N user eye shapes are all a straight line, it is determined that the user has entered a sleep state, and a shutdown control command is issued to control the mobile video playback device to automatically shut down; otherwise, the video playback state of the mobile video playback device is maintained.

Images