Method, system and device for controlling multimedia player and multimedia player

Abstract

The invention discloses a multimedia player control method, system and device and a multimedia player. The method comprises the following steps: acquiring an initial face image by using an image acquisition device; performing preprocessing operation according to the initial face image to obtain a first face feature image; performing correction operation according to the first face feature image to obtain a standard face image; performing feature recognition operation according to the standard face image to obtain a lip opening feature value and a pupil proportion feature value; according to the lip opening characteristic value, when it is judged that the lip opening characteristic value is larger than the opening upper limit value, a volume control instruction is generated, and the volume control instruction is sent to the multimedia player; according to the pupil proportion characteristic value, when it is judged that the pupil proportion characteristic value is smaller than the pupil proportion lower limit value, a picture control instruction is generated, and the picture control instruction is sent to the multimedia player. According to the method, the control convenience and user experience of the multimedia player can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multimedia players, and in particular to a control method, system, device and multimedia player of a multimedia player. Background Art

[0002] At present, with the development of multimedia technology and electronic technology, electronic devices such as computers, mobile phones, and tablets are widely used. In people's daily lives, they need to turn on multimedia devices anytime and anywhere to adjust the sound and picture to enjoy a rich multimedia experience.

[0003] Traditional methods of adjusting sound and picture are mainly remote control, contact buttons, touch buttons, etc. Among them, remote control and contact buttons mostly use physical buttons, which will age over time, causing button failure, and the operating distance and range are limited; while contact buttons or touch buttons have the possibility of contamination, transmission of bacteria and viruses. In addition, for people with limited mobility, contact and touch control are not easy to implement.

[0004] With the development of artificial intelligence technology, face recognition is increasingly widely used, and various smart devices are beginning to use face recognition technology, such as Alipay face payment. As an important recognition technology in biometric technology, face recognition can identify biometrics without requiring users to remember or operate anything. They can identify themselves just by watching normally.

[0005] In the prior art, cameras are mainly used to capture the audience's facial images and transmit them to the facial recognition system. The system uses the facial feature point pattern for positioning, determines the current dynamic features of the audience's face, controls the audio synthesis unit to perform audio synthesis based on the results, and adjusts the picture color.

[0006] Therefore, the existing technology has the problem that the user's usage method is cumbersome, does not conform to the user's natural habits, and is inconvenient to operate. Summary of the invention

[0007] The present invention provides a control method, device, electronic device and storage medium of a multimedia player to achieve volume adjustment and use pupil characteristics as input control quantity, which is more in line with the natural habits of users.

[0008] In a first aspect, in order to solve the above technical problems, the present invention provides a method for controlling a multimedia player, which is characterized by being executed by a server and comprising: Acquiring an initial facial image using an image acquisition device; Performing a preprocessing operation according to the initial facial image to obtain a first facial feature image; Performing a correction operation according to the first facial feature image to obtain a standard facial image; According to the standard facial image, a feature recognition operation is performed to obtain a lip opening characteristic value and a pupil ratio characteristic value; According to the lip opening characteristic value, when it is determined that the lip opening characteristic value is greater than the opening upper limit value, generating a volume control instruction, and sending the volume control instruction to the multimedia player; According to the pupil ratio characteristic value, when it is determined that the pupil ratio characteristic value is less than the pupil ratio lower limit value, a picture control instruction is generated and the picture control instruction is sent to the multimedia player.

[0009] In an optional implementation, the control method of the multimedia player is characterized in that the preprocessing operation is performed according to the initial facial image to obtain the first facial feature image, including: Applying a Gaussian filter or other smoothing filter to smooth the initial facial image to obtain a smoothed facial image; A histogram equalization operation is performed on the smoothed facial image to obtain a first facial feature image.

[0010] In an optional implementation, the control method of the multimedia player is characterized in that the correction operation is performed according to the first facial feature image to obtain a standard facial image, comprising: According to the first facial feature image, a facial left-right symmetry correction operation is performed to obtain a second facial feature image; performing a skin color positioning operation according to the second facial feature image to obtain a facial image; According to the facial image, a facial feature correction operation is performed on the facial features to obtain a standard facial image.

[0011] In an optional implementation, the control method of the multimedia player is characterized in that the feature recognition operation is performed according to the standard facial image to obtain the lip opening characteristic value and the pupil ratio characteristic value, including: According to the standard facial image, a lip shape detection algorithm is performed to obtain lip opening features; When the lip opening feature is determined to be open, the lip shape is calculated according to the standard facial image to obtain a lip opening degree feature value; According to the standard facial image, a pupil detection algorithm is performed to obtain the pupil area and the eye socket area; The pupil is calculated according to the pupil area and the eye socket area to obtain a pupil ratio characteristic value.

[0012] In an optional implementation, the control method of the multimedia player is characterized in that the lip shape is calculated based on the standard facial image to obtain the lip opening characteristic value, and the calculation method is as follows: in, represents the lip opening characteristic value; Indicates the distance between the upper and lower lips; Indicates the length of the face, which is the set value; Indicates preset constants, used to adjust The impact of Represents a preset constant used to adjust the offset of the formula.

[0013] In an optional implementation, the control method of the multimedia player is characterized in that the pupil is calculated based on the pupil area and the eye socket area to obtain a pupil ratio characteristic value, and the calculation method is as follows: in, represents the pupil ratio characteristic value; represents the pupil area; represents the orbital area.

[0014] In an optional implementation, the control method of the multimedia player is characterized in that, according to the standard facial image, a pupil detection algorithm is performed to obtain the pupil area and the eye socket area, and the pupil detection algorithm includes: Performing an image segmentation operation according to the standard facial image to obtain the eye socket position; According to the eye socket position, the number of pixels occupied by the eye socket is calculated to obtain the eye socket area; Performing an image segmentation operation according to the eye position to obtain the pupil position; According to the pupil position, the number of pixels occupied by the pupil is calculated to obtain the pupil area; In a second aspect, a control system of a multimedia player is characterized by comprising: An image acquisition module, used for acquiring an initial facial image; An image preprocessing module, used for performing a preprocessing operation according to the initial facial image to obtain a first facial feature image; An image correction module, used for performing a correction operation based on the first facial feature image to obtain a standard facial image; A feature recognition module, used to perform a feature recognition operation based on the standard facial image to obtain a lip opening characteristic value and a pupil ratio characteristic value; A volume control module, configured to generate a volume control instruction according to the lip opening characteristic value, when determining that the lip opening characteristic value is greater than an opening upper limit value, and send the volume control instruction to the multimedia player; The picture control module is used to generate a picture control instruction according to the pupil ratio characteristic value, when it is determined that the pupil ratio characteristic value is less than the pupil ratio lower limit value, and send the picture control instruction to the multimedia player.

[0015] In a third aspect, the present invention further provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements any one of the above-described methods for controlling a multimedia player when executing the computer program.

[0016] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods for controlling a multimedia player.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a control method for a multimedia player, which is specifically executed by a server. The method first obtains an initial facial image through an image acquisition device, and then preprocesses the image to obtain a first facial feature image. Through further correction operations, it is converted into a standard facial image. Based on the standard facial image, the system performs feature recognition to obtain a lip opening characteristic value and a pupil ratio characteristic value. Based on these characteristic values, when the lip opening characteristic value exceeds a set upper limit, the system generates a volume control instruction and sends it to the multimedia player. When the pupil ratio characteristic value is lower than a preset lower limit, the system generates a corresponding picture control instruction, which is also sent to the player. This method realizes non-contact, biometric-based volume and picture adjustment.

[0018] This control method relies on advanced image processing and feature extraction techniques. By accurately identifying the user's facial expressions and pupil changes, the system can respond in real time. Key technologies include automatic recognition and extraction of facial features, as well as dynamic monitoring of lip opening and pupil ratio. This control mode based on natural facial movements effectively reduces the user's dependence on traditional physical control devices. Its beneficial effect is to improve the user experience, make the control process more intuitive and convenient, and ensure efficient and accurate multimedia control functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1is a schematic flow chart of a method for controlling a multimedia player provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of the structure of a control device of a multimedia player provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] At present, with the development of multimedia technology and electronic technology, electronic devices such as computers, mobile phones, and tablets are widely used. In people's daily lives, they need to turn on multimedia devices anytime and anywhere to adjust the sound and picture to enjoy a rich multimedia experience.

[0021] Traditional methods of adjusting sound and picture are mainly remote control, contact buttons, touch buttons, etc. Among them, remote control and contact buttons mostly use physical buttons, which will age over time, causing button failure, and the operating distance and range are limited; while contact buttons or touch buttons have the possibility of contamination, transmission of bacteria and viruses. In addition, for people with limited mobility, contact and touch control are not easy to implement.

[0022] With the development of artificial intelligence technology, face recognition is increasingly widely used, and various smart devices are beginning to use face recognition technology, such as Alipay face payment. As an important recognition technology in biometric technology, face recognition can identify biometrics without requiring users to remember or operate anything. They can identify themselves just by watching normally.

[0023] In the prior art, cameras are mainly used to capture the audience's facial images and transmit them to the facial recognition system. The system uses the facial feature point pattern for positioning, determines the current dynamic features of the audience's face, controls the audio synthesis unit to perform audio synthesis based on the results, and adjusts the picture color.

[0024] Therefore, the existing technology has the following problems: the volume cannot be adjusted; the pupil is not used as an input to control the volume; the actions of turning the eyes left and right, turning up and down, pressing the tip of the nose towards the eyes, and opening and closing the mouth are used as control instructions, which do not conform to the user's natural habits and are inconvenient to operate.

[0025] In order to solve the above problems, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Reference Figure 1The first embodiment of the present invention provides a method for controlling a multimedia player, comprising the following steps: S11, acquiring an initial facial image using an image acquisition device; S12, performing a preprocessing operation according to the initial facial image to obtain a first facial feature image; S13, performing a correction operation according to the first facial feature image to obtain a standard facial image; S14, performing a feature recognition operation according to the standard facial image to obtain a lip opening characteristic value and a pupil ratio characteristic value; S15, generating a volume control instruction according to the lip opening characteristic value, when it is determined that the lip opening characteristic value is greater than the opening upper limit value, and sending the volume control instruction to the multimedia player; S16, based on the pupil ratio characteristic value, when it is determined that the pupil ratio characteristic value is less than the pupil ratio lower limit value, generating a picture control instruction, and sending the picture control instruction to the multimedia player.

[0027] In step S11, an initial facial image is acquired by using an image acquisition device.

[0028] It should be noted that the initial facial image refers to the user's facial image captured for the first time by the image acquisition device after the system is started. This image contains the full picture of the user's face and can provide basic data for subsequent image processing and feature extraction. The acquisition process is usually completed through a camera or other image acquisition device, which may be installed near the multimedia player, such as above or below the display screen. The choice of image acquisition device depends on the specific scenario. For example, an ordinary RGB camera, CMOS camera, or even a high-resolution infrared camera can be used. In order to enhance the robustness of the system, a multi-camera system is usually used to ensure simultaneous capture from the left and right sides of the user, and to obtain the user's facial images from different angles to ensure that clear and complete facial details can be captured.

[0029] In step S12, the preprocessing operation is performed according to the initial facial image to obtain a first facial feature image, including: Applying a Gaussian filter or other smoothing filter to smooth the initial facial image to obtain a smoothed facial image; A histogram equalization operation is performed on the smoothed facial image to obtain a first facial feature image.

[0030] Among them, the smoothing filter is applied for smoothing processing, aiming to reduce the noise in the image and make the edge and texture of the image softer. Exemplarily, the present invention uses a Gaussian filter for smoothing processing, and smoothes the image by weighted averaging each pixel in the image with its neighboring pixels. Of course, other smoothing filters can also be used to smooth the image, and the present invention is not limited to this.

[0031] It should be noted that in this process, the weight is defined by a Gaussian function, and the farther away from the center pixel, the smaller the effect, so that noise and sharp changes are suppressed. This processing not only helps to eliminate interference caused by uneven light or sensor noise during image acquisition, but also maintains the main structure of facial features, laying the foundation for subsequent processing.

[0032] In specific implementation, when applying Gaussian filtering, you can choose the appropriate filter size and standard deviation (σ) to control the degree of smoothing. A larger filter will bring a more obvious smoothing effect, but may cause some details to be lost, so it needs to be adjusted according to the actual situation of the image.

[0033] It should be noted that the smoothed image is subjected to a histogram equalization operation to further enhance the contrast of the image and obtain a first facial feature image. The purpose of histogram equalization is to make the brightness and contrast of the image more uniform by redistributing the pixel values ​​in the image, especially in the case of poor lighting, so that the facial details can be more prominent. Histogram equalization increases the difference between the bright and dark areas in the image by stretching or compressing the gray value distribution in the image, thereby highlighting the feature points in the image.

[0034] In actual operation, the histogram equalization step can map the pixels in the image to a new grayscale value range by calculating the cumulative distribution function of the image grayscale value. Through this process, facial features such as pupils and mouth contours can be effectively enhanced, allowing the system to more accurately extract these features for subsequent recognition. Histogram equalization is particularly suitable for images with uneven lighting conditions, such as facial images taken in a dark environment. Through equalization, the key areas of the face can be made clearer, ensuring that subsequent feature recognition is accurate and reliable.

[0035] In step S13, the correction operation is performed according to the first facial feature image to obtain a standard facial image, including: According to the first facial feature image, a facial left-right symmetry correction operation is performed to obtain a second facial feature image; performing a skin color positioning operation according to the second facial feature image to obtain a facial image; According to the facial image, a facial feature correction operation is performed on the facial features to obtain a standard facial image.

[0036] It should be noted that the feature points of the facial features in the standard facial image include: the intersection of the left and right facial contours, the intersection of the upper and lower eyelids, the intersection of the upper and lower lips, the intersection between the chin and the corners of the mouth, and other feature point information.

[0037] Among them, the face is first roughly located by skin color detection or feature point detection algorithm, the purpose is to quickly determine the area where the face may exist in the image, and reduce the amount of calculation for subsequent fine processing. Exemplarily, the present invention uses skin color detection to determine the area of ​​the face by analyzing the pixel color. Of course, the face detection method can also use simple feature point detection to roughly frame the facial contour, and the present invention does not limit this.

[0038] It should be noted that the system performs fine positioning within the rough positioning area. Fine positioning further refines the position and posture of the face, and can combine multiple technologies, such as skin color detection and feature point detection, shape model and manifold model, Adaboost algorithm, etc. Exemplarily, the present invention adopts the Adaboost algorithm. Of course, the face position in the image can also be adjusted to conform to the standard facial model to improve the recognition accuracy, which is not limited by the present invention. Among them, the Adaboost algorithm is a classic cascade classifier that can form a strong classifier by continuously superimposing weak classifiers, thereby accurately locating the face.

[0039] It should be noted that the method for obtaining feature points of facial features in a standard facial image. According to the standard facial image, the SURF algorithm operates to generate a scale space of the image. The scale space ensures that key feature points can be detected at different scales, so that the algorithm is robust to images of different distances and sizes. In the scale space, extreme points are located according to the pixel change rate, and these extreme points represent feature points in the image. According to the extreme points, position analysis is performed, and the direction and intensity information of the feature points are obtained through the gradient histogram.

[0040] It should be noted that the method of using the feature point information. The extracted facial features and key angle relationships can be used in a variety of application scenarios, such as facial recognition, sentiment analysis, identity verification, etc. Exemplarily, the present invention uses feature points in the input image to match known feature points in the database to achieve identity recognition. In addition, the system calculates the angle relationship between facial key points, which can determine the user's head posture or emotional expression, further improving the accuracy and richness of recognition.

[0041] In step S14, a feature recognition operation is performed according to the standard facial image to obtain a lip opening characteristic value and a pupil ratio characteristic value, including: According to the standard facial image, a lip shape detection algorithm is performed to obtain lip opening features; When the lip opening feature is determined to be open, the lip shape is calculated according to the standard facial image to obtain a lip opening degree feature value; According to the standard facial image, a pupil detection algorithm is performed to obtain the pupil area and the eye socket area; The pupil is calculated according to the pupil area and the eye socket area to obtain a pupil ratio characteristic value.

[0042] The lip opening characteristic value reflects the degree of opening of the user's lips, and the pupil ratio characteristic value indicates the area ratio of the pupil to the eye socket. These features can indicate the user's intention, such as adjusting the volume by lip opening, or adjusting other screen attributes such as screen brightness and screen size by pupil ratio.

[0043] It should be noted that the lip opening characteristic value is obtained. First, a lip shape detection algorithm is executed based on a standardized facial image. Exemplarily, the present invention adopts a convolutional neural network (CNN). Of course, the shape contour of the mouth can also be detected by traditional image processing methods such as edge detection, which is not limited by the present invention. After the lip area is detected, further analysis will be performed to determine whether the lips are in an open state. When the lip shape is judged to be "open", the lip shape calculation step is entered. The specific calculation method is exemplarily based on geometric analysis, such as by calculating the distance between the upper and lower lips or the area of ​​the mouth contour, to quantify the degree of lip opening, and finally generate a lip opening characteristic value.

[0044] It should be noted that the pupil ratio characteristic value is obtained. First, the pupil detection algorithm is performed. Exemplarily, the present invention determines the boundary of the eye through grayscale analysis, and further distinguishes the pupil and the white of the eye area, and calculates the area of ​​the pupil and the area of ​​the entire eye socket. Next, the ratio of the pupil area to the eye socket area is calculated to obtain the pupil ratio characteristic value. This ratio value can reflect the changes in the user's pupil under different lighting environments, and can also serve as an indication of the user's emotions or attention.

[0045] It should be noted that the lip opening characteristic value and the pupil ratio characteristic value are used. When the system detects that the lip opening characteristic value exceeds the preset threshold, the system will generate a volume adjustment instruction. For example, the greater the opening, the more the volume increases; when the lips are closed or the opening is less than a certain value, the system will lower the volume. Similarly, the pupil ratio characteristic value can be used to adjust the brightness or color of the picture. When the pupil ratio is less than the set lower limit, it indicates that the ambient light is strong, and the system will generate an instruction to adjust the screen brightness to reduce the screen brightness and ensure the comfort of the user's viewing experience.

[0046] Exemplarily, the range of pupil ratio is 0-1, and when the pupil ratio is between 0-0.2, the corresponding picture adjustment instruction is to execute the picture reduction or closing instruction; when the pupil ratio is between 0.2-0.3, the corresponding picture adjustment instruction is to execute the picture non-adjustment instruction; when the pupil ratio is between 0.3-0.7, the corresponding picture adjustment instruction is to execute the picture enlargement instruction; when the pupil ratio is between 0.7-1, the corresponding picture adjustment instruction is to execute the picture pause instruction. In another implementation scheme, the range of pupil ratio is 0-1, and when the pupil ratio is between 0-0.1, the corresponding picture adjustment instruction is to execute the picture reduction or closing instruction; when the pupil ratio is between 0.1-0.15, the corresponding picture adjustment instruction is to execute the picture non-adjustment instruction; when the pupil ratio is between 0.15-0.6, the corresponding picture adjustment instruction is to execute the picture enlargement instruction; when the pupil ratio is between 0.6-1, the corresponding picture adjustment instruction is to execute the picture pause instruction. Of course, the interval setting of picture adjustment can be adjusted according to the actual scene, and the same is true for volume adjustment, and the present invention does not limit this.

[0047] In step S15, according to the lip opening characteristic value, when it is determined that the lip opening characteristic value is greater than the opening upper limit value, a volume control instruction is generated and sent to the multimedia player, wherein the lip opening characteristic value is calculated as follows: in, represents the lip opening characteristic value; Indicates the distance between the upper and lower lips; Indicates the length of the face, which is the set value; Indicates preset constants, used to adjust The impact of Represents a preset constant used to adjust the offset of the formula.

[0048] In step S16, according to the lip opening characteristic value, when it is determined that the lip opening characteristic value is greater than the opening upper limit value, a volume control instruction is generated and sent to the multimedia player, wherein the pupil ratio characteristic value calculation method is as follows: in, represents the pupil ratio characteristic value; represents the pupil area; represents the orbital area.

[0049] It should be noted that the pupil area and the orbital area are calculated using a method.

[0050] Performing an image segmentation operation according to the standard facial image to obtain the eye socket position; According to the eye socket position, the number of pixels occupied by the eye socket is calculated to obtain the eye socket area; Performing an image segmentation operation according to the eye position to obtain the pupil position; According to the pupil position, the number of pixels occupied by the pupil is calculated to obtain the pupil area; In order to facilitate the understanding of the present invention, some preferred embodiments of the present invention are further described below.

[0051] In one implementation, the system first obtains the user's initial facial image through an image acquisition device, and uses a multi-CMOS camera, which is placed directly above the multimedia player, to capture the user's left and right facial images. The image is transmitted to the subsequent processing module for analysis. The system smoothes the acquired image and eliminates noise through a Gaussian filter to make the image edge smoother. Then, histogram equalization is performed to adjust the image contrast to make the facial features more obvious, thereby generating a standard facial image for feature extraction.

[0052] On this basis, the system performs feature recognition operations for lip opening and pupil ratio. First, the system performs a lip shape detection algorithm based on the standardized facial image to determine whether the lips are open. If the mouth corner opening is less than or equal to 0.1, the system generates a play instruction to start or continue playing the media content; if the mouth corner opening is greater than 0.1, the system further performs corresponding operations based on the pupil ratio.

[0053] For pupil ratio detection, the system uses a standard facial image to identify the pupil and eye socket, calculates the ratio of pupil area to eye socket area, and thus obtains the pupil ratio feature value. If the pupil ratio is less than 0.5 and the mouth opening is greater than 0.1, the system generates a volume adjustment instruction. When the volume is less than 50%, the volume adjustment instruction is to increase the volume; when the volume is greater than 50%, the volume adjustment instruction is to reduce the volume. If the pupil ratio is greater than or equal to 0.5, the system will not generate a volume adjustment instruction and maintain the current volume.

[0054] Next, the system further executes the screen adjustment command based on the pupil ratio. The system sets a preset threshold of 0.4. If the user's pupil ratio is between 0.3 and 0.4 and the eyes are looking straight ahead, the system will not adjust the screen and maintain the original display state; if the pupil ratio is between 0 and 0.3, the system generates a screen reduction command to reduce the screen display; when the pupil ratio is between 0.4 and 0.5, the system generates a screen enlargement command; if the pupil ratio is greater than or equal to 0.5, a pause screen display command is generated to stop the screen playback. When the pupil ratio is less than or equal to 0.2, the system executes a close command to close the media content.

[0055] In some embodiments, the system also combines the detection of pupil emmetropia to further refine the control. If the user's mouth opening is less than or equal to 0.1, and the eyeballs are looking straight ahead, and the pupil emmetropia is decreasing, the system generates a mute command; if the eyeballs are not looking straight ahead at this time, and the pupil emmetropia is increasing, the system generates a pause command. Similarly, when the system detects that the pupil ratio is between 0.4 and 0.5, if the user's eyes are squinting and the pupil emmetropia is increasing, the system generates a play command; if the eyeballs are not looking straight ahead and the pupil emmetropia is decreasing, the system generates a command to lower the volume.

[0056] In another implementation, a sound source localization operation may be added. Exemplarily, in this embodiment, the present invention proposes a multimedia player control method based on the combination of sound source localization and audio and video content recognition, aiming to provide users with a more intelligent and natural control method through sound source localization and audio analysis.

[0057] First, the system uses sound source localization technology to capture the user's sound source location through the sound source localization module and transmit the location signal to the subsequent processing module. This process accurately determines the user's sound source location by analyzing the arrival time difference and intensity of the sound, providing a data basis for subsequent audio analysis. In this way, the system can obtain the user's real-time location and enhance its understanding of the user's intentions.

[0058] Next, the system enters the audio analysis step. At this stage, the audio analysis module receives the user's voice input and filters and samples it to form a time domain signal. Thereafter, the time domain signal is converted into a frequency domain signal through methods such as Fourier transform, wavelet transform, and short-time Fourier transform. Based on the time domain signal and the frequency domain signal, the Mel cepstral coefficient is extracted to obtain a feature signal. Based on the feature signal, it is transmitted to the back-end processing module for subsequent face recognition and content recognition.

[0059] After the audio analysis, the system proceeds to the video content recognition step. First, the feature signal obtained from the audio analysis is preprocessed, and then the video content feature parameters are extracted. The system compares these feature parameters with the pre-stored video content feature database to determine whether the current video content belongs to the feature category in the database, such as film and television programs, games, advertisements, tutorials, or exams. If the recognition result belongs to the feature in the database, the system transmits the comparison result to the back-end processing module; if not, no transmission is performed.

[0060] In addition, the system also has a mode switching step, allowing users to flexibly select the input source. When the mode switching switch signal is triggered, the system selects the image input module or the audio analysis module as the next input source according to the user's intention. If the mode switching switch signal indicates the use of the image input module, the image content is recognized; if it indicates the use of the audio analysis module, the audio content is recognized.

[0061] The following describes the working process of the present invention using a common scenario as an example. Figure 2 , which is Figure 1 Schematic diagram of the working scenario of the method.

[0062] In a typical home scenario, the user is watching a movie played by a multimedia player in the living room, and the system automatically enters the intelligent control mode, ready to operate through the user's voice and facial expressions. First, the system captures the user's facial image through the image acquisition device module installed above the multimedia player. Multiple CMOS cameras capture the user's facial image from the left and right sides respectively, ensuring that the user's expression and eye pupil changes are fully captured. At the same time, the player's microphone array locates the sound source and confirms the user's sound source location. By analyzing the time difference and sound intensity of the sound source, the system can accurately determine the user's position in the room, helping subsequent image and sound processing modules focus on the user.

[0063] After obtaining the initial image of the user, the system's image preprocessing module preprocesses the image. First, the image is smoothed by a Gaussian filter or other smoothing filter to reduce the noise in the image. Then, after the histogram equalization operation, the image contrast is enhanced and the facial features are clearer. Through the image correction module, the system generates a standard facial image, and then uses a special lip detection algorithm and pupil detection algorithm to extract the user's lip opening and pupil ratio feature values. Through the detection of lip opening, the system can recognize the user's expression and determine whether the user is issuing a voice command or wants to control the volume. Through the feature recognition module, the lip opening feature value and the pupil ratio feature value are obtained.

[0064] At the same time, when the user speaks a voice command, the microphone captures the user's voice signal and filters and samples it, converting it into a time domain signal. Subsequently, the system converts the time domain signal into a frequency domain signal through algorithms such as Fourier transform or wavelet transform. The system extracts the user's voice feature parameters from it, especially through the Mel cepstral coefficients. The system can analyze the time-frequency energy changes and amplitudes in the voice, and these parameters are further used for face recognition or multimedia player control decisions.

[0065] After processing the image and audio signals, the system starts to make intelligent control decisions through the volume control module and the picture control module. For example, during the movie playback, if the user sends a signal to adjust the volume through voice or facial expression, the system first detects the user's lip opening. If the user's mouth corner opening is less than or equal to 0.1, the system directly executes the playback instruction; when the mouth corner opening is greater than 0.1, the system further determines the volume adjustment operation through the pupil ratio. If the pupil ratio is less than 0.5, the system generates a volume adjustment instruction, which is adjusted according to the current volume. When the volume is less than 50%, the system automatically increases the volume; when the volume is greater than 50%, the system lowers the volume. If the user's pupil ratio is greater than or equal to 0.5, no volume adjustment operation is performed.

[0066] In this scenario, users can also control the size of the screen by changing their pupils. When the user's pupil ratio is between 0.3 and 0.4, the system keeps the screen unchanged; when the pupil ratio is less than 0.3, the system automatically reduces the screen; and when the pupil ratio is between 0.4 and 0.5, the system enlarges the screen. If the pupil ratio is greater than 0.5, the system performs a pause operation to pause the currently playing content. In addition, if the pupil ratio is less than 0.2, the system directly closes the multimedia player.

[0067] Throughout the process, the user's natural interaction is not limited to voice and pupil changes. The system also incorporates a mode switching function, allowing users to seamlessly switch between different control modes. For example, users can switch the system's input source through voice input and choose to control the player through the image input module or the audio analysis module. During playback, the system can also determine whether to switch modes based on the user's facial expressions or pupil changes. When the user's eyes are facing the screen and the pupil's emmetropia increases, the system may resume playback; if the eyes are squinting and the emmetropia decreases, the system may pause playback, providing a more intelligent and flexible multimedia control experience.

[0068] In summary, the present invention provides a control method for a multimedia player, which is specifically executed by a server. The method first obtains an initial facial image through an image acquisition device, and then preprocesses the image to obtain a first facial feature image. Through further correction operations, it is converted into a standard facial image. On the basis of the standard facial image, the system performs feature recognition to obtain a lip opening characteristic value and a pupil ratio characteristic value. Based on these characteristic values, when the lip opening characteristic value exceeds a set upper limit, the system generates a volume control instruction and sends it to the multimedia player. When the pupil ratio characteristic value is lower than a preset lower limit, the system generates a corresponding picture control instruction, which is also sent to the player. This method realizes contactless, biometric-based volume and picture adjustment.

[0069] This control method relies on advanced image processing and feature extraction techniques. By accurately identifying the user's facial expressions and pupil changes, the system can respond in real time. Key technologies include automatic recognition and extraction of facial features, as well as dynamic monitoring of lip opening and pupil ratio. This control mode based on natural facial movements effectively reduces the user's dependence on traditional physical control devices. Its beneficial effect is to improve the user experience, make the control process more intuitive and convenient, and ensure efficient and accurate multimedia control functions.

[0070] Reference Figure 2 A second embodiment of the present invention provides a multimedia player device, comprising: An image acquisition module, used for acquiring an initial facial image; An image preprocessing module, used for performing a preprocessing operation according to the initial facial image to obtain a first facial feature image; An image correction module, used for performing a correction operation based on the first facial feature image to obtain a standard facial image; A feature recognition module, used to perform a feature recognition operation based on the standard facial image to obtain a lip opening characteristic value and a pupil ratio characteristic value; A volume control module, configured to generate a volume control instruction according to the lip opening characteristic value, when determining that the lip opening characteristic value is greater than an opening upper limit value, and send the volume control instruction to the multimedia player; The picture control module is used to generate a picture control instruction according to the pupil ratio characteristic value, when it is determined that the pupil ratio characteristic value is less than the pupil ratio lower limit value, and send the picture control instruction to the multimedia player.

[0071] Preferably, the image acquisition module uses a multi-sensor collaborative working method to collect the left and right facial images of the user through two or more CMOS cameras. These cameras are placed directly above the multimedia player, and the bottom corresponds to the position of the speaker, ensuring that the user's facial expressions and pupil changes can be captured at the same time. Through such multi-view capture, the system can analyze the user's facial features more comprehensively and accurately, and provide more refined data support for subsequent volume and picture adjustments.

[0072] It should be noted that a multimedia player device provided in an embodiment of the present invention is used to execute all process steps of a multimedia player method in the above embodiment, and the working principles and beneficial effects of the two correspond one to one, so they will not be repeated.

[0073] The embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a program. When the processor executes the computer program, the steps in the above-mentioned various multimedia control method embodiments are implemented, such as Figure 1 Alternatively, when the processor executes the computer program, the functions of the modules / units in the above-mentioned device embodiments are realized, such as the image correction module, the feature recognition module and other modules.

[0074] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0075] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than the above components, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0076] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0077] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the electronic device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0078] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0079] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0080] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for controlling a multimedia player, characterized in that: Executed by the server, including: Acquiring an initial facial image using an image acquisition device; Performing a preprocessing operation according to the initial facial image to obtain a first facial feature image; Performing a correction operation according to the first facial feature image to obtain a standard facial image; According to the standard facial image, a feature recognition operation is performed to obtain a lip opening characteristic value and a pupil ratio characteristic value; According to the lip opening characteristic value, when it is determined that the lip opening characteristic value is greater than the opening upper limit value, generating a volume control instruction, and sending the volume control instruction to the multimedia player; According to the pupil ratio characteristic value, when it is determined that the pupil ratio characteristic value is less than the pupil ratio lower limit value, a picture control instruction is generated and the picture control instruction is sent to the multimedia player.

2. The control method of the multimedia player according to claim 1, characterized in that: The step of performing a preprocessing operation on the initial facial image to obtain a first facial feature image includes: Applying a Gaussian filter or other smoothing filter to smooth the initial facial image to obtain a smoothed facial image; A histogram equalization operation is performed on the smoothed facial image to obtain a first facial feature image.

3. The control method of the multimedia player according to claim 1, characterized in that: The step of performing a correction operation according to the first facial feature image to obtain a standard facial image includes: According to the first facial feature image, a facial left-right symmetry correction operation is performed to obtain a second facial feature image; performing a skin color positioning operation according to the second facial feature image to obtain a facial image; According to the facial image, a facial feature correction operation is performed on the facial features to obtain a standard facial image.

4. The control method of the multimedia player according to claim 1, characterized in that: The step of performing a feature recognition operation according to the standard facial image to obtain a lip opening characteristic value and a pupil ratio characteristic value includes: According to the standard facial image, a lip shape detection algorithm is performed to obtain lip opening features; When the lip opening feature is determined to be open, the lip shape is calculated according to the standard facial image to obtain a lip opening degree feature value; According to the standard facial image, a pupil detection algorithm is performed to obtain the pupil area and the eye socket area; The pupil is calculated according to the pupil area and the eye socket area to obtain a pupil ratio characteristic value.

5. The control method of the multimedia player according to claim 4, characterized in that: The lip shape is calculated based on the standard facial image to obtain the lip opening characteristic value, and the calculation method is as follows: in, represents the lip opening characteristic value; Indicates the distance between the upper and lower lips; Indicates the length of the face, which is the set value; Indicates preset constants, used to adjust The impact of Represents a preset constant used to adjust the offset of the formula.

6. The control method of the multimedia player according to claim 4, characterized in that: The pupil is calculated based on the pupil area and the eye socket area to obtain the pupil ratio characteristic value, and the calculation method is as follows: in, represents the pupil ratio characteristic value; represents the pupil area; The orbital area.

7. The control method of the multimedia player according to claim 4, characterized in that: The pupil detection algorithm is performed according to the standard facial image to obtain the pupil area and the eye socket area, and the pupil detection algorithm includes: Performing an image segmentation operation according to the standard facial image to obtain the eye socket position; According to the eye socket position, the number of pixels occupied by the eye socket is calculated to obtain the eye socket area; Performing an image segmentation operation according to the eye socket position to obtain the pupil position; According to the pupil position, the number of pixels occupied by the pupil is calculated to obtain the pupil area.

8. A control system for a multimedia player, characterized in that: include: An image acquisition module, used for acquiring an initial facial image; An image preprocessing module, used for performing a preprocessing operation according to the initial facial image to obtain a first facial feature image; An image correction module, used for performing a correction operation based on the first facial feature image to obtain a standard facial image; A feature recognition module, used to perform a feature recognition operation based on the standard facial image to obtain a lip opening characteristic value and a pupil ratio characteristic value; A volume control module, configured to generate a volume control instruction according to the lip opening characteristic value, when determining that the lip opening characteristic value is greater than an opening upper limit value, and send the volume control instruction to the multimedia player; The picture control module is used to generate a picture control instruction according to the pupil ratio characteristic value, when it is determined that the pupil ratio characteristic value is less than the pupil ratio lower limit value, and send the picture control instruction to the multimedia player.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the control method of the multimedia player as claimed in any one of claims 1 to 7 when executing the computer program.

10. A multimedia player, characterized in that: The multimedia player is used to communicate with the control system of the multimedia player as claimed in claim 8 and receive picture control instructions.

Images