Ring volume adjusting method and device, electronic equipment and storage medium

Abstract

The invention provides a ringing volume adjusting method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of terminal control. The ringing volume adjusting method comprises the following steps: when a ringing event occurs, acquiring a face image; identifying the face image to obtain user identity information and sight line identification information; and adjusting the ringing volume according to the user identity information and the sight line identification information. According to the invention, the flexibility of ringing volume adjustment can be improved, automatic adjustment of the ringing volume is realized, and the user experience is improved.

Description

Background Art

[0002] At present, electronic devices are becoming more and more popular. Communication messages may be received in various scenarios, such as incoming calls, video calls, voice calls, etc. Users are often required to press the volume button to adjust the ringing volume when the ringing occurs, or select the vibration mode or silent mode in the settings in advance to control the ringing switch. The above methods cannot achieve automatic adjustment of the ringing volume, lack flexibility, and affect the user's volume experience. Summary of the invention

[0003] The purpose of the present disclosure is to provide a ring volume adjustment method, a ring volume adjustment device, an electronic device and a computer-readable storage medium, thereby improving the flexibility of ring volume adjustment at least to a certain extent and realizing automatic adjustment of the ring volume.

[0004] According to a first aspect of the present disclosure, a method for adjusting the ringing volume is provided, comprising: when a ringing event occurs, collecting a face image; identifying the face image to obtain user identity information and line of sight recognition information; and adjusting the ringing volume according to the user identity information and line of sight recognition information.

[0005] According to a second aspect of the present disclosure, a ring volume adjustment device is provided, comprising: an image acquisition module, for acquiring a facial image when a ring event occurs; an image recognition module, for recognizing the facial image to obtain user identity information and line of sight recognition information; and a volume adjustment module, for adjusting the ring volume according to the user identity information and line of sight recognition information.

[0006] According to a third aspect of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to implement the above method by executing the executable instructions.

[0007] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

[0008] The ringing volume adjustment method provided in the embodiment of the present disclosure, on the one hand, collects a face image when the ringing occurs, and recognizes the face image to obtain user identity information and line of sight recognition information, and then adjusts the ringing volume according to the user identity information and line of sight recognition information, thereby avoiding the user from actively adjusting the ringing volume when the ringing occurs, thereby realizing automatic adjustment of the ringing volume; on the other hand, the ringing volume is adjusted in combination with the user identity information and line of sight recognition information. The ringing volume is not a fixed volume, but is related to the user identity and the user line of sight, making the setting of the ringing volume more flexible and user-adaptable, so that the user can get an excellent volume experience.

[0009] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0011] Figure 1 A schematic diagram of a related art for ringing adjustment is shown.

[0012] Figure 2 A schematic diagram showing another related technology for ringing adjustment is shown.

[0013] Figure 3 The following schematically shows a flow chart of a ring volume adjustment method in an exemplary embodiment of the present disclosure.

[0014] Figure 4 A schematic diagram of collecting facial images in an exemplary embodiment of the present disclosure is schematically shown.

[0015] Figure 5 A schematic diagram of an interface for selecting a target user in an exemplary embodiment of the present disclosure is schematically shown.

[0016] Figure 6 A flowchart schematically illustrates an implementation method for determining a first volume adjustment multiple in an exemplary embodiment of the present disclosure.

[0017] Figure 7 A schematic diagram schematically illustrates different distances between a user and a screen and corresponding face areas in an exemplary embodiment of the present disclosure.

[0018] Figure 8 A flowchart schematically illustrates another implementation method of determining a first volume adjustment multiple in an exemplary embodiment of the present disclosure.

[0019] Fig. 9 A flowchart schematically illustrates an implementation method for determining an effective duration of a first volume adjustment multiple in an exemplary embodiment of the present disclosure.

[0020] Fig.10 A flowchart schematically illustrates an implementation method for automatically adjusting the effective duration corresponding to the first volume adjustment multiple in an exemplary embodiment of the present disclosure.

[0021] Fig.11A flowchart schematically illustrates an implementation method of adjusting a first volume adjustment multiple within an effective duration in an exemplary embodiment of the present disclosure.

[0022] Fig.12 A schematic diagram schematically illustrates a method for determining an effective duration of a second volume adjustment multiple in an exemplary embodiment of the present disclosure.

[0023] Fig.13 A flow chart schematically illustrates a ring volume adjustment method based on a volume setting event in an exemplary embodiment of the present disclosure.

[0024] Fig.14 A flowchart for determining sight line identification information in an exemplary embodiment of the present disclosure is schematically shown.

[0025] Fig.15 A structural schematic diagram of a sight line estimation model to which the exemplary embodiments of the present disclosure can be applied is shown.

[0026] Fig.16 A flowchart of performing line of sight estimation based on a line of sight estimation model in an exemplary embodiment of the present disclosure is shown.

[0027] Fig.17 A schematic diagram showing a facial posture feature to which an exemplary embodiment of the present disclosure can be applied.

[0028] Fig.18 A complete flow chart of a ring volume adjustment method in an exemplary embodiment of the present disclosure is shown.

[0029] Fig.19 The figure shows a schematic diagram of the composition of a ring volume adjustment device to which the exemplary embodiments of the present disclosure can be applied.

[0030] Fig. 20 The figure shows a schematic diagram of the composition of an electronic device to which the exemplary embodiments of the present disclosure can be applied. DETAILED DESCRIPTION

[0031] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0032] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0033] See also Figure 1 A schematic diagram of ringing adjustment in the related art includes a ringing option 101, a vibration option 102 and a silent option 103. The user can select the vibration option 102 or the silent option 103 to remind the user by vibration or silent (no ringing) when a ringing event occurs. Figure 1 The figure also includes a volume adjustment option 104 for adjusting the volume in the ringing state. The user can adjust the volume by moving the sliding bar of the volume adjustment option 104.

[0034] See also Figure 2 This is another schematic diagram of the related art for adjusting the ringing volume. The user can press the volume adjustment button 201 on the terminal to adjust the ringing volume, and the adjusted volume obtained by pressing the adjustment button can be simultaneously presented on the user interface.

[0035] However, in Figure 1 and Figure 2 In the ringing adjustment method shown, the ringing volume still depends on the user's active setting. On the one hand, the user needs to frequently switch between various ringing modes. On the other hand, the user may ignore the adjustment of the ringing volume, resulting in a bad volume experience for the user when a ringing event occurs in many scenarios. For example, the ringing in important scenarios at work, study or life (such as meetings and classes) causes interference to other users or the user himself.

[0036] From this, it can be known how to automatically adjust the ringing volume when a ringing event occurs, so that the ringing volume neither affects others nor hinders one's own user experience. Based on one or more of the above problems, the embodiments of the present disclosure provide a ringing volume adjustment method, which collects a face image when a ringing event occurs, and identifies the face image to obtain user identity information and line of sight recognition information, and then adjusts the ringing volume according to the user identity information and line of sight recognition information. This can avoid the user actively adjusting the ringing volume when the ringing occurs, and realize automatic adjustment of the ringing. In addition, the ringing volume is not fixed, but is related to the user identity and the user's line of sight, so that the setting of the ringing volume is more flexible and user-adaptable.

[0037] In the embodiment of the present disclosure, the provided ring volume adjustment method can be executed by a terminal device. In the case where all steps in the ring volume adjustment method provided in the embodiment of the present disclosure can be executed by the terminal device, all steps can be executed by a processor of the terminal device. The terminal device can be an intelligent device with a data processing function, such as a smart phone, a computer, a tablet computer, a vehicle-mounted device, a wearable device, a monitoring device, a television, etc. The terminal device can also be called a mobile terminal, a terminal, a mobile device, etc. The present disclosure does not limit the type of the terminal device.

[0038] Furthermore, the ring volume adjustment method provided in the embodiment of the present disclosure can also be executed by a server. Correspondingly, in this mode of execution by the server, the server can start executing the steps in the technical solution of the embodiment of the present disclosure in response to a trigger instruction, wherein the trigger instruction can be sent by a terminal device used by the user, or can be triggered locally by the server in response to some automation events. The server can be a background system that provides related services in the embodiment of the present disclosure, and can include a portable computer, a desktop computer, a smart phone, and other electronic devices with computing functions, or a cluster formed by multiple electronic devices.

[0039] In addition, the ring volume adjustment method of the embodiment of the present disclosure can also be executed collaboratively by the terminal device and the server. In this way of collaborative execution by the electronic device and the server, some steps in the ring volume adjustment method provided by the embodiment of the present disclosure are executed by the terminal device, while other steps are executed by the server. For example, the ring volume adjustment method provided by the embodiment of the present disclosure can be performed by the terminal device collecting a facial image when a ring event occurs, and sending the facial image to the server, and the server identifies the facial image to obtain user identity information and line of sight recognition information, and sends the user identity information and line of sight recognition information to the terminal device, and then the terminal device adjusts the ring volume according to the user identity information and line of sight recognition information.

[0040] It should be noted that in this method in which the terminal device and the server are executed collaboratively, the steps executed by the terminal device and the server respectively can be dynamically adjusted according to actual conditions, and there is no special restriction on this.

[0041] In the embodiments of the present disclosure, the technical solution is executed by a terminal device as an example for description.

[0042] Figure 3 A flow chart of a ring volume adjustment method according to an exemplary embodiment of the present disclosure is shown. Figure 3 The ring volume adjustment method of the embodiment of the present disclosure may include steps S310 to S330:

[0043] In step S310, when a ringing event occurs, a face image is collected.

[0044] In the exemplary embodiments of the present disclosure, a ringing event refers to an event that causes a terminal device to ring, including but not limited to a voice call, a video call, a voice call, an alarm, a text message, a reminder, etc. When a ringing event occurs, the front camera of the terminal device can be called to capture the current image.

[0045] The current picture may be an image including a face or an image not including a face. If the current picture does not include a face, the process returns to the image acquisition stage until a face image is obtained and the process proceeds to the next step.

[0046] For example, see Figure 4 A schematic diagram of collecting face images according to an exemplary embodiment of the present disclosure is shown. When the ringing starts, image frames are collected, for example, at intervals of 50ms (or other time intervals), and face detection is performed. When image frame 4 includes a face, the face image recognition step is started, otherwise, image frames are continuously collected.

[0047] It should be noted that the embodiments of the present disclosure may utilize a face detection algorithm to perform face detection on the collected image frames. The face detection algorithm includes but is not limited to MTCNN (Multi-Task Convolutional Neural Network), Retina Face (a face recognition algorithm), SSD (Single Shot MultiBoxDetector), etc. The embodiments of the present disclosure do not specifically limit the specific type of the face detection algorithm.

[0048] In step S320, the face image is recognized to obtain user identity information and sight line recognition information.

[0049] In an exemplary embodiment of the present disclosure, user identity information is used to indicate the current user identity, such as whether the current user is the corresponding owner of the terminal device, or whether the current user is another non-owner user designated by the owner. Line of sight recognition information is used to indicate whether the current user's line of sight is looking at the terminal device.

[0050] Recognition of facial images includes face recognition and line of sight recognition, wherein a face recognition algorithm can be used to recognize a face image, that is, to calculate and analyze the position of the facial features, face shape and angle, and then compare it with an existing face template to determine the true identity. Face recognition algorithms include but are not limited to recognition algorithms based on facial feature points, recognition algorithms based on the entire face image, recognition algorithms based on templates, and algorithms using neural networks for recognition, such as LBP (Local Binary Patterns), etc. The disclosed embodiments do not specifically limit the specific types of face recognition algorithms.

[0051] Among them, in order to make the collected face image meet the processing requirements of the face recognition algorithm, the face recognition algorithm can be preprocessed. For example, the face image can be aligned first to ensure that the eye line is in a horizontal state, and then the eye distance is adjusted to the required size (such as 50 pixels, which is determined according to the image resolution requirements of different face recognition algorithms), and finally the face image is scaled to the size required by the face recognition algorithm. Of course, the embodiments of the present disclosure can also include other preprocessing processes, such as light compensation, grayscale conversion, normalization, filtering, sharpening, etc., which are not limited to this.

[0052] Line of sight recognition can use a line of sight estimation algorithm to identify and process a face image to obtain line of sight recognition information. The line of sight estimation algorithm extracts eye features (such as the direction of eye gaze, the position of the gaze point), facial posture features, etc., and performs line of sight estimation based on the extracted features. Of course, the embodiment of the present disclosure can also use the placement state of the terminal device as an input feature, and use the line of sight estimation algorithm to perform line of sight estimation, thereby increasing the reference factors for line of sight estimation.

[0053] In step S330, the ringing volume is adjusted according to the user identity information and the line of sight recognition information.

[0054] In an exemplary embodiment of the present disclosure, the ringing volume can be turned up or down based on the user identity information and the line of sight estimation information, so that the ringing volume is not a fixed volume, but is related to the user identity and the user line of sight, making the setting of the ringing volume more flexible and user-adaptable.

[0055] The implementation details of the above steps are described in detail below.

[0056] Usually, when using a terminal device, in order to avoid the ringing volume being too loud to disturb other users or the user himself, the ringing volume needs to be lowered in time. In an exemplary embodiment, a method for adjusting the ringing volume is provided. Adjusting the ringing volume according to the user identity information and the line of sight recognition information includes:

[0057] In response to the user identity information and the line of sight recognition information satisfying a first condition, a first volume adjustment multiple is determined, where the first volume adjustment multiple is a positive number less than 1.

[0058] The first volume adjustment factor is used to adjust the current volume of the ringing sound, that is, the ringing volume is determined according to the first volume adjustment factor and the current volume. The current volume of the ringing sound is the ringing volume before adjustment, which can be set in advance by the user or can be the default ringing volume of the terminal device. Since the first volume adjustment factor is a positive number less than 1, the ringing volume can be lowered based on the first volume adjustment factor.

[0059] In the embodiment of the present disclosure, the user identity information and the sight line recognition information satisfying the first condition include:

[0060] The user identity information matches the identity of the target user, and the line of sight recognition information indicates that the user is looking at the screen; wherein the target user is the main user or a backup user corresponding to the main user.

[0061] Specifically, the match between the user identity information and the target user's identity indicates that the current user is a trusted user, and the line of sight recognition information indicates that the user is looking at the screen, which means that the user is consciously looking at the terminal device, that is, the user has learned about the occurrence of the ringing event, and can even learn the detailed details of the ringing event, such as the caller number, video call object, etc.

[0062] The facial image of the owner of the machine can be entered in advance. When determining the user's identity information, the recognition result of the collected facial image is compared with the facial image of the owner of the machine. If the comparison is successful, the user identity information is determined to be the owner of the machine, and then it can be determined that the user identity information matches the identity of the target user; otherwise, it is determined to be a non-owner user.

[0063] In some possible embodiments, the owner user may also designate his backup user as the target user, that is, the fact that the user identity information matches the owner user's backup user also indicates that the user identity information matches the target user's identity.

[0064] Among them, the owner of the device can select an image as a backup user in the gallery of the terminal device, or the owner of the device can take pictures of other users through the terminal device and use the captured image for the identity matching process, or the owner's facial information that has been entered can be directly called for the identity matching process, such as facial information for unlocking the device, facial information for logging into the application, etc.

[0065] For example, Figure 5 FIG. 2 shows a schematic diagram of an interface for selecting a target user according to an exemplary embodiment of the present disclosure. Figure 5After the user selects the option of automatically adjusting the incoming call volume, the interface presents selection prompt information and corresponding selection function options, which are used to instruct the user to enter the target user image for automatically adjusting the incoming call volume.

[0066] After the user clicks on the "Confirm" selection function option, the interface jumps to the shooting interface and provides a gallery function option for selecting images, allowing the user to take pictures by themselves or select images from an existing gallery. Then, the image corresponding to the target user is determined based on the image taken or selected by the user for use in the identity matching process.

[0067] By taking the owner user or the backup user corresponding to the owner user as the target user, when the owner user is inconvenient or is not next to the terminal device when a ringing event occurs, the backup user designated by the owner user can complete the identity matching. Since the backup user designated by the owner user is usually a user trusted by the owner user, the owner user will usually be prompted in time after the identity matching is completed. In this case, there is no need for an excessively loud ringing volume. Therefore, it can be determined that the user identity information and the line of sight recognition information meet the first condition, and the first volume adjustment multiple is determined, which is used to automatically reduce the ringing volume to avoid excessively loud ringing volume.

[0068] The disclosed embodiment can determine a first volume adjustment factor for reducing the ringing volume when the user identity information matches the target user and the line of sight recognition information indicates that the user is looking at the screen. That is, the ringing volume can be reduced by the target user looking at the screen of the terminal device without the need for manual operation by the user.

[0069] In an exemplary embodiment, before determining the first volume adjustment multiple in response to the user identity information and the sight line recognition information satisfying the first condition, the following may also be performed:

[0070] Get the current volume corresponding to the ringing event and determine whether the current volume is higher than the set volume threshold.

[0071] The set volume threshold refers to the volume at which the ringing volume will not interfere with other users or the user himself, such as 30% of the maximum volume of the system as the volume threshold. When the ringing volume at the time of the ringing event is lower than the volume threshold, it means that the ringing volume will not interfere with others or the user himself, and the process of automatically adjusting the ringing volume can be omitted.

[0072] On the contrary, if the ringing volume when the ringing event occurs is higher than the set volume threshold, the process of automatically adjusting the ringing volume is entered, that is, determining whether the user identity information and the line of sight recognition information meet the first condition.

[0073] Based on this, the ringing volume does not need to be adjusted according to the first volume adjustment multiple in all cases. It is only adjusted when it is considered that the ringing volume has adverse noise interference. This reduces the computing energy consumption of the device and makes the ringing volume adjustment more flexible.

[0074] In an exemplary embodiment, a method for determining a first volume adjustment factor is provided. Figure 6 As shown, in response to the user identity information and the sight recognition information satisfying the first condition, determining the first volume adjustment multiple may include steps S610 to S630:

[0075] Step S610: Acquire the face area in the face image.

[0076] In the process of detecting a face image, the face frame coordinate information, the face key points (such as eyes, nose tip, mouth corners) coordinate information, etc. can be obtained, wherein the face area can be determined based on the face frame coordinate information.

[0077] Step S620: Obtaining the area ratio of the face area to the screen display area.

[0078] Usually, the size of the face area can reflect the distance between the user and the screen. The first volume adjustment multiple varies depending on the distance between the user and the screen.

[0079] like Figure 7 A schematic diagram is shown of different distances between the user and the screen. When the user is 0.5 meters away from the screen, the area of ​​the face region in the face image is smaller, and when the user is 0.2 meters away from the screen, the area of ​​the face region in the face image is larger. The disclosed embodiment measures the distance between the user and the screen by obtaining the area ratio of the face region to the screen display area, which eliminates the influence of different screen sizes of the terminal devices used by the user to a certain extent, so that the area ratio more accurately reflects the distance between the user and the screen.

[0080] Step S630: determining a first volume adjustment multiple according to the area ratio.

[0081] In the disclosed embodiment, the correspondence between the area ratio and the first volume adjustment multiple can be pre-set, wherein the area ratio can be negatively correlated with the first volume adjustment multiple, that is, the larger the area ratio, the smaller the first volume adjustment multiple. Since the first volume adjustment multiple is a positive number less than 1, the ringing volume can be reduced to a greater extent based on the smaller first volume adjustment multiple.

[0082] See also Figure 7As shown in the figure, when the user faces the screen (0.2 meters away from the screen), the area ratio is large, indicating that the user is looking at the screen at a relatively close distance. At this time, there is no need for an excessively loud ringing volume. The smaller the first volume adjustment multiple is, the greater the ringing volume is. On the contrary, when the user is far away from the screen (0.5 meters away), in order to achieve the effect of ringing reminder, the ringing volume is reduced less.

[0083] The disclosed embodiment determines a first volume adjustment multiple based on the area ratio of the face area to the screen display area to achieve automatic adjustment of the volume adjustment multiple, and further enables automatic adjustment of the ringing volume when a ringing event occurs, so that the ringing volume can neither affect others nor hinder one's own user experience.

[0084] In an exemplary embodiment, another implementation method for determining the first volume adjustment multiple is also provided. Figure 8 As shown, in response to the user identity information and the sight line recognition information satisfying the first condition, determining the first volume adjustment multiple may include step S810 and step S820:

[0085] Step S810: Determine the duration of the user's gaze on the screen based on the gaze recognition information.

[0086] Step S820: determining a first volume adjustment factor according to the length of time the user stares at the screen.

[0087] The duration of the user's gaze on the screen can be determined with the ringing event as the starting point, and the first volume adjustment multiple can be presented in real time on the device screen, so that the user can accurately know the target multiple that needs to be adjusted.

[0088] By determining the first volume adjustment multiple based on the length of time the user stares at the screen, the user can avoid performing additional operations such as pressing the volume button, thereby reducing the operational complexity and thinking cost of adjusting the ringing volume, and improving the volume adjustment efficiency.

[0089] Usually, users are not always in a situation where they need to lower the ringing volume. To prevent users from missing the opportunity to respond to the ringing event, in an exemplary embodiment, a method for determining the effective duration of the first volume adjustment multiple is also provided.

[0090] like Fig. 9 As shown, in response to the user identity information and the sight line recognition information satisfying the first condition, determining the first volume adjustment multiple may further include step S910 and step S920:

[0091] Step S910: Acquire the first user status information when the ringing event occurs.

[0092] The first user status information may include at least one of the scene information corresponding to the user, the sight state information, and the device holding state information. The scene information corresponding to the user, such as the meeting scene, the class scene, the sleeping scene, etc., can be determined based on the collected images and / or the ambient sound; the sight state information, such as the sight direction, the sight angle, etc., can be determined based on the collected images; the device holding state includes holding and not holding (placing), which can be determined based on the sensor information of the terminal device, such as gravity acceleration information, gyroscope information, etc. The disclosed embodiment does not impose any special restrictions on the specific method of obtaining the first user status information.

[0093] Step S920: determining the effective duration corresponding to the first volume adjustment multiple according to the first user status information.

[0094] The effective duration corresponding to the first volume adjustment multiple refers to adjusting the ringing volume according to the first volume adjustment multiple for a ringing event occurring within the effective duration.

[0095] A mapping relationship between different first user status information and effective durations may be preset, and the effective duration corresponding to the first volume adjustment multiple may be determined based on the mapping relationship. When the first user status information includes two or more types of information (such as scene information and device holding status information), the effective duration may be obtained based on each type of information, and the effective duration of the first volume adjustment multiple may be determined based on the effective duration corresponding to each type of information.

[0096] Exemplarily, if the first user status information includes scene information and device holding state information, the first effective duration can be determined based on the mapping relationship between the scene information and the effective duration, and the second effective duration can be determined based on the mapping relationship between the device holding state information and the effective duration, and the effective duration of the first volume adjustment multiple is determined in combination with the first effective duration and the second effective duration. For example, the first effective duration and the second effective duration are weighted and summed based on a preset weight. Of course, the embodiments of the present disclosure may also use other methods to combine the effective durations corresponding to each information to determine the effective duration of the first volume adjustment multiple, including but not limited to the above-mentioned methods.

[0097] It should be noted that after determining the effective time corresponding to the first volume adjustment multiple, within the effective time, regardless of whether the user is looking at the screen, the ringing event occurring in the terminal device will adjust the ringing volume according to the first volume adjustment multiple, thereby ensuring that no ringing that interferes with other users or the user himself will occur within a period of time, thereby avoiding the user from frequently performing operations of looking at the screen.

[0098] Considering that the user may have a ringing event that needs to be responded to within the effective duration, in order to further improve the accuracy of the effective duration, in an exemplary embodiment, the effective duration can also be adjusted in response to an adjustment operation on the effective duration.

[0099] Specifically, the effective duration corresponding to the first volume adjustment multiple and the corresponding adjustment function item can be presented on the screen, and the user can confirm the effective duration by operating the adjustment function item, or shorten / extend the effective duration by adjusting the function item. The user can adjust the effective duration by touching the adjustment function item through a touch medium (such as a finger), or can adjust the effective duration based on the viewpoint position of the user looking at the screen (such as looking at a sliding bar for adjusting the effective duration). The embodiments of the present disclosure include but are not limited to the above-mentioned methods for adjusting the effective duration.

[0100] In an exemplary embodiment, a method for automatically adjusting the effective duration corresponding to the first volume adjustment multiple is also provided. Fig.10 , determining the effective duration corresponding to the first volume adjustment multiple according to the first user status information, further comprising steps S1010 and S1020:

[0101] Step S1010: obtaining the second user status information according to a preset period within the effective time;

[0102] Step S1020: adjusting the effective duration based on the second user status information and the first user status information.

[0103] The second user status information and the first user status information are user status information at different time points, so the second user status information may also include at least one of the scene information corresponding to the user, the sight state information, and the device holding state information. The scene information corresponding to the user is the scene information corresponding to the user collected by the terminal, and the scene information corresponding to the user can be determined based on the information collected by the terminal, such as performing scene recognition based on the images collected by the terminal, the ambient sound and other information to obtain the scene information corresponding to the user.

[0104] During the effective duration of the first volume adjustment multiple, the second user status information can be obtained according to a preset period to adjust the effective duration based on the second user status information and the first user status information. The preset period can be set according to actual needs, or it can be a default period, and there is no special limitation on this. Among them, the effective duration can be extended or shortened based on the second user status information and the first user status information.

[0105] Specifically, adjusting the effective duration based on the second user status information and the first user status information may include: first acquiring a status similarity between the second user status information and the first user status information, and then adjusting the effective duration according to the status similarity.

[0106] The degree of state similarity is used to characterize the degree of state similarity of users at different times. Taking the user state information including scene information as an example, if the second user state information is a meeting scene, which is consistent with the first user state information (meeting scene), it means that the user is continuously in a meeting, so the effective duration of the first volume adjustment multiple for reducing the ringing volume can be extended. Correspondingly, if the second user state information is a dining scene, which is very different from the meeting scene of the first user state information, even if it is still within the effective duration, and the scene in which the user is located has changed significantly, the effective duration can be shortened. The process of adjusting the effective duration based on other user state information is similar and will not be repeated here.

[0107] Among them, a mapping relationship between the similarity degree of different states and the time adjustment amount can be preset, so as to determine the time adjustment amount for adjusting the effective duration based on the mapping relationship.

[0108] It should be noted that if the user status information includes two or more types of information (such as scene information and device holding status information), the time adjustment amount for adjusting the effective time can be determined according to each information, and the effective time can be adjusted according to the time adjustment amount corresponding to each information.

[0109] In addition, the disclosed embodiment can also collect multiple second user status information multiple times according to a preset period within the effective duration, and obtain the state similarity between each second user status information and the first user status information, to obtain multiple state similarities. The effective duration of the first volume adjustment multiple is adjusted only when there are results corresponding to state similarities exceeding the target number indicating that the effective duration needs to be extended (or shortened). In other words, by comparing the state changes of users at different times multiple times, it is further confirmed whether the effective duration is extended or shortened, thereby improving the accuracy of adjusting the effective duration.

[0110] The disclosed embodiment automatically adjusts the effective duration of the first volume adjustment multiple based on user status information at different times within the effective duration, so that the duration of reducing the ringing volume adapts to the user's status changes and conforms to the user's habits or behaviors. It is more flexible and avoids the user's frequent switching of the ringing volume in different states, providing a better user experience.

[0111] In an exemplary embodiment, within the effective duration for adjusting the ringing volume based on the first volume adjustment multiple, that is, the volume of the ringing event within the effective duration is reduced. In order to avoid the situation where the user is unable to respond to the ringing event in time within the effective duration, an implementation method for adjusting the first volume adjustment multiple within the effective duration is also provided.

[0112] like Fig.11 After determining the effective duration of the first volume adjustment multiple, the method further includes steps S1110 and S1120:

[0113] Step S1110: Acquire the captured images within the effective time period, and identify the captured images.

[0114] During the effective time, the front camera of the terminal device can be called to capture the current picture, and face detection and face recognition can be performed on the captured image. The method of performing face detection and face recognition can be found in the relevant records of step S310 and step S320, which will not be repeated here.

[0115] Step S1120: If the recognition result of the captured image does not meet the first condition, the moment corresponding to the first volume adjustment multiple is determined as the time starting point, and after a preset time period, the first volume adjustment multiple is switched to a second volume adjustment multiple, and the second volume adjustment multiple is a number greater than or equal to 1; wherein the preset time period is less than the effective time period.

[0116] The recognition result of the collected image does not meet the first condition including the following situations: the collected image does not contain a face, the user identity information corresponding to the collected image does not match the identity of the target user, and the sight line recognition information corresponding to the collected image indicates that the user is not looking at the screen.

[0117] When the recognition result of the captured image does not meet the first condition, the user may not respond to the ringing event after the ringing event occurs and the user looks at the screen (the first volume adjustment multiple is determined), such as leaving the terminal device. Therefore, in order to remind the user to respond to the ringing event in time, the ringing volume can be increased after a preset period of time, that is, the first volume adjustment multiple is switched to the second volume adjustment multiple.

[0118] The preset duration may be a default duration or may be set according to actual conditions, and there is no special limitation on this.

[0119] In a possible embodiment, after the first volume adjustment multiple is switched to the second volume adjustment multiple, the second volume adjustment multiple will continue the effective duration of the first volume adjustment multiple. For example, if the effective duration of the first volume adjustment multiple is 20 minutes, the effective duration of the second volume adjustment multiple will be 20 minutes.

[0120] In another possible embodiment, after the first volume adjustment multiple is switched to the second volume adjustment multiple, part of the remaining effective duration of the first volume adjustment multiple may be used as the effective duration of the second volume adjustment multiple. Fig.12 As shown, if the validity period of the first volume adjustment multiple is 30 minutes, and the first volume adjustment multiple is switched to the second volume adjustment multiple at the 10th minute, the validity period of the second volume adjustment multiple can be from the 10th minute to the 20th minute, and the second volume adjustment multiple is switched back to the first volume adjustment multiple after the 20th minute, thereby avoiding noise caused by prompting the user with a high ringing volume for too long.

[0121] The disclosed embodiment prevents the user from missing the opportunity to respond to the ringing event by switching the first volume adjustment multiple within the effective time of the first volume adjustment multiple.

[0122] Generally, lowering the ringing volume can avoid disturbing other users or the user himself when a ringing event occurs. However, considering that the ringing can also be used as a prompt sound to find a device or protect user privacy, the embodiment of the present disclosure also provides a method for increasing the ringing volume.

[0123] Adjust the ring volume based on user identity information and line of sight recognition information, and also include:

[0124] In response to the user identity information and the line of sight recognition information satisfying the second condition, a third volume adjustment multiple is determined, and the third volume adjustment multiple is a number greater than 1.

[0125] The third volume adjustment factor is used to adjust the current volume of the ringing sound, that is, the ringing volume is determined according to the third volume adjustment factor and the current volume. Since the third volume adjustment factor is a number greater than 1, the ringing volume can be increased based on the third volume adjustment factor.

[0126] In the embodiment of the present disclosure, the user identity information and the sight line recognition information satisfy the second condition, including:

[0127] The user identity information does not match the identity of the target user, and the line of sight recognition information indicates that the user is looking at the screen; wherein the target user is the owner user or a backup user corresponding to the owner user.

[0128] Specifically, if the user identity information does not match the identity of the target user, it means that the current user is an untrustworthy user, and the sight recognition information indicates that the user is looking at the screen, which means that the user is consciously looking at the terminal device. In this case, in order to protect the user's privacy, the ringing volume can be increased to remind the target user.

[0129] In an exemplary embodiment, in order to further improve the accuracy of adjusting the ring volume, the user identity information and the line of sight recognition information satisfying the second condition further include:

[0130] The presence of preset operation behaviors is detected, and the preset operation behaviors include irregular operation behaviors on the device without authorization.

[0131] Preset operation behaviors include but are not limited to: frequent unlocking errors, irregular pressing of function keys, forced shutdown and other unauthorized irregular operation behaviors. When the preset operation behaviors are detected at the same time, it is further confirmed that an untrusted user is operating the terminal device when the ringing event occurs. Then, a warning can be issued by increasing the ringing volume, which also makes it easier for the owner to find the device.

[0132] For example, if the owner finds that the phone is lost, when calling the phone, the phone recognizes that the non-target user is looking at the screen and there is a preset operation, and the ring volume can be increased in time, thereby helping the owner to quickly find the phone.

[0133] When the user identity information and the line of sight recognition information meet the second condition, the embodiment of the present disclosure increases the ringing volume by determining a third volume adjustment multiple, thereby providing convenience for protecting user privacy and device security and improving the user's usage experience.

[0134] In the embodiment of the present disclosure, the user can freely choose whether to turn on the vector volume automatic adjustment. Figure 5 As shown, if the user does not turn on automatic ringing volume adjustment, when a ringing event occurs, the ringing volume is not automatically adjusted, and the ringing is performed at the default volume or the user-defined volume.

[0135] When the user turns on automatic ring volume adjustment, in order to achieve automatic adjustment of the ring volume without affecting the user experience, a ring volume adjustment method based on volume setting events is also provided, such as Fig.13 As shown, the process may include:

[0136] Step S1310: Acquire device setting events within a preset time period before the ringing event occurs.

[0137] Device setting events include, but are not limited to, ring type setting events, volume setting events, network setting events, screen time setting events, display setting events, wallpaper setting events, etc. The preset time period can be a default time period or can be set according to actual needs, and there is no special restriction on this. Among them, the volume setting event refers to setting the volume of the ring, such as the default volume or the custom volume.

[0138] Step S1320: adjusting the ringing volume based on whether there is a volume setting event in the device setting event.

[0139] If it is determined that there is no volume setting event in the device setting event, it means that the user currently has no special ringing volume requirement, and then the embodiment of the present disclosure can be executed to adjust the ringing volume according to the user identity information and the line of sight recognition information.

[0140] On the contrary, if there is a volume setting event within the preset time period before the ringing event occurs, the volume corresponding to the volume setting event is used as the ringing volume. In other words, the user has performed a volume setting operation (such as increasing or decreasing the ringing volume) before the ringing event occurs, and a ringing volume that meets the user's expectations already exists. Therefore, when the ringing event occurs, the volume set by the user himself is used as the ringing volume, and the ringing volume is not adjusted according to the user identity information and the line of sight recognition information.

[0141] If there are multiple volume setting events of the user within a preset time period, the volume corresponding to the last volume setting event before the ringing event occurs is used as the ringing volume.

[0142] The disclosed embodiment detects volume setting events to make the adjustment of the ring volume more in line with the user's personalized needs, thereby obtaining a better volume experience and improving the user's usage experience.

[0143] The following describes the process of obtaining the line of sight recognition information according to the embodiment of the present disclosure.

[0144] See also Fig.14 FIG. 4 shows a flow chart of determining sight line identification information according to an exemplary embodiment of the present disclosure. Fig.14 , the process includes steps S1410 to S1430:

[0145] Step S1410: Detect the face image to obtain a face detection result.

[0146] The face image is detected using a face detection algorithm to obtain face frame coordinate information, face key point coordinate information, etc. The disclosed embodiment does not specifically limit the type of face detection algorithm.

[0147] Step S1420: Obtain a left-eye sub-image and a right-eye sub-image.

[0148] The left eye sub-image and the right eye sub-image may be collected separately, or the left eye sub-image and the right eye sub-image may be obtained based on the face image. The sizes of the left eye sub-image and the right eye sub-image may be determined according to actual needs, and no special restrictions are imposed on this.

[0149] Step S1430: Determine the sight line recognition information based on the left eye sub-image, the right eye sub-image and the face detection result.

[0150] The disclosed embodiment may utilize a gaze estimation model to perform gaze estimation on a left eye sub-image, a right eye sub-image, and a face detection result to obtain gaze recognition information.

[0151] Among them, Fig.15 A structural diagram of a sight line estimation model according to an exemplary embodiment of the present disclosure is shown. The sight line estimation model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a feature fusion module and a feature decision module.

[0152] Among them, the first feature extraction module is used to extract the first human eye feature related to the gaze direction and the gaze point coordinates in the left eye sub-image, and the second feature extraction module is used to extract the second human eye feature related to the gaze direction and the gaze point coordinates in the right eye sub-image. The first feature extraction module and the second feature extraction module can both include multiple convolution layers and pooling layers (sampling layers). The embodiment of the present disclosure does not limit the specific number of convolution layers and pooling layers. The third feature extraction module is used to extract the facial posture features in the face detection results, and can include multiple fully connected layers, such as two fully connected layers. The feature fusion module may include a concat layer (for connecting human eye features and facial posture features) and multiple fully connected layers.

[0153] The embodiment of the present disclosure does not limit the specific number of layers of each network layer. The feature decision module may include a fully connected layer for indicating whether the user is looking at the screen.

[0154] like Fig.16 A flowchart of performing sight line estimation based on a sight line estimation model according to an exemplary embodiment of the present disclosure is shown, which may include steps S1610 to S1650:

[0155] Step S1610: using a first feature extraction module to extract features from the left eye sub-image to obtain a first human eye feature.

[0156] Step S1620: using a second feature extraction module to extract features from the right eye sub-image to obtain second human eye features.

[0157] Step S1630: Based on the face detection result, a third feature extraction module is used to extract posture features to obtain face posture features.

[0158] Face posture features include but are not limited to pitch angle, yaw angle, and roll angle. Fig.17 , the head (face) rotates around the Y axis as the pitch angle, the head (face) rotates around the X axis as the yaw angle, and the head (face) rotates around the Z axis as the roll angle. Based on the facial posture features, the facial posture information is provided for the line of sight estimation, so that the line of sight estimation introduces the facial posture factor to improve the accuracy of the line of sight estimation result.

[0159] Step S1640: The first human eye feature, the second human eye feature and the facial posture feature are fused through a feature fusion module to obtain a fused feature.

[0160] Step S1650: The feature decision module is used to estimate the line of sight of the fused features to obtain line of sight recognition information.

[0161] The gaze estimation model based on the embodiment of the present disclosure integrates binocular features and facial posture features, and has high gaze estimation accuracy. The gaze estimation result that can be obtained is whether the user is looking at the screen or not looking at the screen.

[0162] It should be noted that the embodiment of the present disclosure can also pre-train the gaze estimation model based on sample data. Each sample data includes a binocular sub-image sample and a corresponding face detection result sample, as well as label data (whether the user is looking at the screen). The gaze estimation model is trained using the sample data until the training stop condition is obtained.

[0163] Fig.18 A complete flow chart of a ring volume adjustment method according to an exemplary embodiment of the present disclosure is shown. Fig.18 , the ring volume adjustment method of the embodiment of the present disclosure is described by taking the ring event being a voice call as an example.

[0164] Step S1810: The user turns on the automatic ring volume adjustment function and enters facial information.

[0165] Among them, the user can directly capture the face image through the camera, or the user can select the face image from the existing gallery; or, after the user turns on the automatic ring volume adjustment function, the recorded face information of the owner of the phone is called, such as the face information used to unlock the device, the face information used to log in to the application, etc.

[0166] Specifically, after the user selects the option of automatically adjusting the incoming call volume, options for entering facial information are provided, such as providing a gallery function option for the user to select facial images, providing a function option for obtaining shooting permissions to enter facial images, and providing a function option for obtaining calling permissions to call the facial information that has been entered. The user can select the corresponding function option to enter facial information according to actual needs.

[0167] Step S1820: When a voice call comes in, the terminal device collects a facial image when the call rings.

[0168] If no face is detected in the current image being collected, the ring volume is not adjusted, and image collection continues until a face image is collected, and the process proceeds to step S1830.

[0169] Step S1830: Recognize the facial image collected when the ringing sound is ringing to obtain user identity information and line of sight recognition information.

[0170] Step S1840: Determine whether the current user is the target user based on the user identity information.

[0171] If the current user is determined to be the target user, it is further determined whether the current user is looking at the screen. Further, if the user identity information and the sight line recognition information meet the first condition, that is, the user identity information matches the identity of the target user, and the sight line recognition information indicates that the user is looking at the screen, then go to step S1850; if the user identity information and the sight line recognition information meet the second condition, that is, the user identity information does not match the identity of the target user, the sight line recognition information indicates that the user is looking at the screen, and the presence of a preset operation behavior is detected, then go to step S1860.

[0172] Step S1850: Determine a first volume adjustment multiple.

[0173] The effective duration of the first volume adjustment multiple may also be determined, and for a ringing event occurring within the effective duration, the ringing volume is adjusted according to the first volume adjustment multiple.

[0174] During the effective time, the effective time can also be adjusted, and the first volume adjustment multiple within the effective time can be switched to the second volume adjustment multiple, etc.

[0175] After determining the first volume adjustment multiple, go to step S1870.

[0176] Step S1860: Determine a third volume adjustment multiple.

[0177] After determining the third volume adjustment multiple, the ringing volume is adjusted based on the third volume adjustment multiple.

[0178] Step S1870: Determine whether there is a volume setting event within a preset time period before the voice call.

[0179] If there is a volume setting event within a preset time period before the voice call, the volume corresponding to the volume setting event is used as the ringing volume. Otherwise, the ringing volume is adjusted based on the first volume adjustment multiple.

[0180] It should be noted that when the effective time period ends, if there is a ringing event, the above steps S1810 to S1870 are repeated, which will not be described in detail. In addition, the specific details of the above steps S1810 to S1870 have been described in detail in the above embodiments, and the undisclosed details can be referred to the contents of the above exemplary embodiments, which will not be described here.

[0181] From the above content, it can be seen that the ringing volume adjustment method provided in the embodiment of the present disclosure, on the one hand, collects a facial image when the ringing occurs, and recognizes the facial image to obtain user identity information and line of sight recognition information, and then adjusts the ringing volume according to the user identity information and line of sight recognition information, thereby avoiding the user from actively adjusting the ringing volume when the ringing occurs and avoiding setting the ringing mode in advance, thereby realizing automatic adjustment of the ringing; on the other hand, the ringing volume is adjusted in combination with the user identity information and line of sight recognition information. The ringing volume is not a fixed volume, but is related to the user identity and the user's line of sight, making the setting of the ringing volume more flexible and user-adaptable, so that the user can get an excellent volume experience.

[0182] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0183] For further reference, Fig.19 As shown, in an exemplary embodiment of the present disclosure, a ring volume adjustment device 1900 is provided, including an image acquisition module 1910, an image recognition module 1920 and a volume adjustment module 1930, wherein:

[0184] The image acquisition module 1910 is used to collect facial images when a ringing event occurs; the image recognition module 1920 is used to recognize the facial image and obtain user identity information and line of sight recognition information; the volume adjustment module 1930 is used to adjust the ringing volume according to the user identity information and line of sight recognition information.

[0185] In an exemplary embodiment, the volume adjustment module 1930 is configured to execute: in response to the user identity information and the line of sight recognition information satisfying a first condition, determining a first volume adjustment multiple, the first volume adjustment multiple being a positive number less than 1.

[0186] In an exemplary embodiment, the user identity information and the line of sight recognition information satisfy a first condition, including: the user identity information matches the identity of the target user, and the line of sight recognition information indicates that the user is looking at the screen; wherein the target user is the owner user or a backup user corresponding to the owner user.

[0187] In an exemplary embodiment, the volume adjustment module 1930 is further configured to execute: obtaining a current volume corresponding to the ringing event, and determining whether the current volume is higher than a set volume threshold.

[0188] In an exemplary embodiment, the volume adjustment module 1930 is configured to execute: obtaining a face area in a face image; obtaining an area ratio of the face area to a screen display area; and determining the first volume adjustment multiple according to the area ratio.

[0189] In an exemplary embodiment, the volume adjustment module 1930 is configured to perform: determining the length of time the user stares at the screen based on the gaze recognition information; determining a first volume adjustment factor based on the length of time the user stares at the screen; wherein the first volume adjustment factor is presented on the device screen.

[0190] In an exemplary embodiment, the volume adjustment module 1930 is also configured to perform: obtaining the first user status information when a ringing event occurs; determining the effective duration corresponding to the first volume adjustment multiple based on the first user status information; wherein, for a ringing event occurring within the effective duration, adjusting the ringing volume based on the first volume adjustment multiple.

[0191] In an exemplary embodiment, the first user status information includes at least one of scene information corresponding to the user, line of sight status information, and device holding status information.

[0192] In an exemplary embodiment, the volume adjustment module 1930 is configured to execute: in response to an adjustment operation on the effective duration, adjusting the effective duration.

[0193] In an exemplary embodiment, the volume adjustment module 1930 is further configured to execute: obtaining the second user status information according to a preset period within the effective time; and adjusting the effective time based on the second user status information and the first user status information.

[0194] In an exemplary embodiment, the volume adjustment module 1930 is further configured to execute: obtaining the state similarity between the second user state information and the first user state information; and adjusting the effective time according to the state similarity.

[0195] In an exemplary embodiment, the volume adjustment module 1930 is further configured to execute: acquiring a captured image within a valid duration and identifying the captured image; if the identification result of the captured image does not meet the first condition, determining the moment corresponding to the first volume adjustment multiple as the time starting point, and after a preset duration, switching the first volume adjustment multiple to a second volume adjustment multiple, the second volume adjustment multiple being a number greater than or equal to 1; wherein the preset duration is less than the valid duration.

[0196] In an exemplary embodiment, the volume adjustment module 1930 is configured to execute: in response to the user identity information and the line of sight recognition information satisfying the second condition, determining a third volume adjustment multiple, the third volume adjustment multiple being a number greater than 1.

[0197] In an exemplary embodiment, the user identity information and the line of sight identification information satisfy the second condition, including: the user identity information does not match the identity of the target user, and the line of sight identification information indicates that the user is looking at the screen; wherein the target user is the owner user or a backup user corresponding to the owner user.

[0198] In an exemplary embodiment, the user identity information and the line of sight recognition information satisfy the second condition, and further include: detecting the presence of a preset operation behavior, the preset operation behavior including irregular operation of the device without authorization.

[0199] In an exemplary embodiment, the volume adjustment module 1930 is further configured to execute: obtaining device setting events within a preset time period before the ringing event occurs; and determining that there is no volume setting event in the device setting events.

[0200] In an exemplary embodiment, the volume adjustment module 1930 is further configured to execute: if there is a volume setting event within a preset time period before the ringing event occurs, the volume corresponding to the volume setting event is used as the ringing volume.

[0201] In an exemplary embodiment, the image recognition module 1920 is configured to perform: detecting a face image to obtain a face detection result; acquiring a left eye sub-image and a right eye sub-image; and determining line of sight recognition information based on the left eye sub-image, the right eye sub-image and the face detection result.

[0202] In an exemplary embodiment, the image recognition module 1920 is configured to execute: using a line of sight estimation model to perform line of sight estimation on the left eye sub-image, the right eye sub-image and the face detection result to obtain line of sight recognition information; wherein the line of sight estimation model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a feature fusion module and a feature decision module; the process of performing line of sight estimation includes: using the first feature extraction module to perform feature extraction on the left eye sub-image to obtain a first human eye feature; using the second feature extraction module to perform feature extraction on the right eye sub-image to obtain a second human eye feature; based on the face detection result, using the third feature extraction module to perform posture feature extraction to obtain facial posture features; using the feature fusion module to fuse the first human eye feature, the second human eye feature and the facial posture feature to obtain a fused feature; performing line of sight estimation on the fused feature through the feature decision module to obtain line of sight recognition information.

[0203] The specific details of each module in the above-mentioned terminal have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, that is, the explanation and beneficial effects of the ring volume adjustment method of the above-mentioned embodiment are also applicable to the ring volume adjustment device 1900 of the embodiment of the present disclosure, and will not be elaborated here.

[0204] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0205] The exemplary embodiments of the present disclosure also provide an electronic device for the above method, which may be the above imaging device or server. Generally, the electronic device includes at least a processor and a memory, the memory is used to store executable instructions of the processor, and the processor is configured to execute the above method by executing the executable instructions.

[0206] Below Fig. 20 Taking the mobile terminal 2000 in the example, the structure of the electronic device in the embodiment of the present disclosure is exemplarily described. It should be understood by those skilled in the art that in addition to the components specifically used for mobile purposes, Fig. 20 The structure in the figure can also be applied to fixed type devices. In other embodiments, the mobile terminal 2000 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware. The interface connection relationship between the components is only shown schematically and does not constitute a structural limitation of the mobile terminal 2000. In other embodiments, the mobile terminal may also adopt the same Fig. 20 Different interface connection methods, or a combination of multiple interface connection methods.

[0207] like Fig. 20 As shown, the mobile terminal 2000 may specifically include: a processor 2001, a memory 2002, a bus 2003, a mobile communication module 2004, an antenna 1, a wireless communication module 2005, an antenna 2, a display screen 2006, a camera module 2007, an audio module 2008, a power module 2009, and a sensor module 2010.

[0208] Processor 2001 may include one or more processing units, for example, processor 2001 may include AP (Application Processor), modem processor, GPU (Graphics Processing Unit), ISP (Image Signal Processor), controller, encoder, decoder, DSP (Digital Signal Processor), baseband processor and / or NPU (Neural-Network Processing Unit), etc.

[0209] The encoder can encode (i.e. compress) an image or video to reduce the data size for easy storage or transmission. The decoder can decode (i.e. decompress) the encoded data of an image or video to restore the image or video data. The mobile terminal 2000 can support one or more encoders and decoders, for example: image formats such as JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), BMP (Bitmap), and video formats such as MPEG (Moving Picture Experts Group) 1, MPEG10, H.1063, H.1064, and HEVC (High Efficiency Video Coding).

[0210] The processor 2001 may be connected to the memory 2002 or other components via a bus 2003 .

[0211] The memory 2002 can be used to store computer executable program codes, which include instructions. The processor 2001 executes various functional applications and data processing of the mobile terminal 2000 by running the instructions stored in the memory 2002. The memory 2002 can also store application data, such as images, videos and other files.

[0212] The communication function of the mobile terminal 2000 can be realized by the mobile communication module 2004, antenna 1, wireless communication module 2005, antenna 2, modulation and demodulation processor and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 2004 can provide 3G, 4G, 5G and other mobile communication solutions applied to the mobile terminal 2000. The wireless communication module 2005 can provide wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. applied to the mobile terminal 2000.

[0213] The display screen 2006 is used to implement display functions, such as displaying user interfaces, images, videos, etc., and displaying abnormal prompt information. The camera module 2007 is used to implement shooting functions, such as shooting images, videos, etc., to collect scene images. The audio module 2008 is used to implement audio functions, such as playing audio, collecting voice, etc. The power module 2009 is used to implement power management functions, such as charging the battery, powering the device, monitoring the battery status, etc. The sensor module 2010 may include one or more sensors to implement corresponding induction detection functions.

[0214] In addition, the exemplary embodiments of the present disclosure further provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of the present specification.

[0215] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0216] In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0217] In addition, program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0218] Other embodiments of the present disclosure will be readily apparent to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and embodiments are to be considered merely as exemplary, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A ring volume adjustment method, characterized in that: include: When a ringing event occurs, a face image is collected; Recognize the facial image to obtain user identity information and sight line recognition information; The ring volume is adjusted according to the user identity information and the line of sight recognition information.

2. The method according to claim 1, characterized in that The adjusting the ring volume according to the user identity information and the sight recognition information includes: In response to the user identity information and the line of sight recognition information satisfying a first condition, a first volume adjustment multiple is determined, where the first volume adjustment multiple is a positive number less than 1.

3. The method according to claim 2, characterized in that The user identity information and the sight line recognition information satisfy a first condition, including: The user identity information matches the identity of the target user, and the sight line identification information indicates that the user is looking at the screen; The target user is the main user or a backup user corresponding to the main user.

4. The method according to claim 2, characterized in that: Before determining the first volume adjustment multiple in response to the user identity information and the sight recognition information satisfying the first condition, the method further includes: A current volume corresponding to the ringing event is obtained, and it is determined whether the current volume is higher than a set volume threshold.

5. The method according to claim 2, characterized in that: In response to the user identity information and the sight recognition information satisfying a first condition, determining a first volume adjustment multiple includes: Acquire a face region in the face image; Obtaining the area ratio of the face area to the screen display area; The first volume adjustment multiple is determined according to the area ratio.

6. The method according to claim 2, characterized in that The method further comprises: Acquire the first user status information when the ringing event occurs; Determine the effective duration corresponding to the first volume adjustment multiple according to the first user status information; Wherein, for a ringing event occurring within the effective time, the ringing volume is adjusted according to the first volume adjustment multiple; The first user status information includes at least one of scene information corresponding to the user, line of sight status information, and device holding status information.

7. The method according to claim 6, characterized in that The method further comprises: During the effective time, obtaining the second user status information according to a preset period; The effective duration is adjusted based on the second user status information and the first user status information.

8. The method according to claim 7, characterized in that The adjusting the effective duration based on the second user status information and the first user status information includes: Acquire the similarity between the second user status information and the first user status information; The effective duration is adjusted according to the similarity of the states.

9. The method according to claim 6, characterized in that The method further comprises: Acquire the collected images within the effective time, and identify the collected images; If the recognition result of the collected image does not meet the first condition, the moment corresponding to the first volume adjustment multiple is determined as the time starting point, and after a preset time, the first volume adjustment multiple is switched to a second volume adjustment multiple, where the second volume adjustment multiple is a number greater than or equal to 1; The preset duration is shorter than the effective duration.

10. The method according to claim 1, characterized in that The adjusting the ring volume according to the user identity information and the sight recognition information includes: In response to the user identity information and the line of sight recognition information satisfying a second condition, a third volume adjustment multiple is determined, and the third volume adjustment multiple is a number greater than 1.

11. The method according to claim 10, characterized in that The user identity information and the sight line identification information satisfy the second condition, including: The user identity information does not match the identity of the target user, and the sight line identification information indicates that the user is looking at the screen; The target user is the main user or a backup user corresponding to the main user.

12. The method according to claim 11, characterized in that The user identity information and the sight line recognition information satisfy the second condition, further comprising: The presence of a preset operation behavior is detected, where the preset operation behavior includes irregular operation behavior on the device without authorization.

13. The method according to any one of claims 1 to 12, characterized in that Recognizing the face image to obtain the sight line recognition information includes: Detecting the face image to obtain a face detection result; Get the left eye sub-image and the right eye sub-image; The sight line recognition information is determined based on the left eye sub-image, the right eye sub-image and the face detection result.

14. The method according to claim 13, characterized in that The determining the sight line recognition information based on the left eye sub-image, the right eye sub-image and the face detection result includes: Using a sight line estimation model to perform sight line estimation on the left eye sub-image, the right eye sub-image and the face detection result to obtain the sight line recognition information; Wherein, the sight line estimation model includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a feature fusion module and a feature decision module; The process of line of sight estimation includes: Using the first feature extraction module to extract features from the left eye sub-image to obtain a first human eye feature; Using the second feature extraction module to extract features from the right eye sub-image to obtain a second human eye feature; Based on the face detection result, using the third feature extraction module to extract posture features to obtain face posture features; The first human eye feature, the second human eye feature and the facial posture feature are fused by the feature fusion module to obtain a fused feature; The line of sight is estimated on the fused feature through the feature decision module to obtain the line of sight recognition information.

15. A ring volume adjustment device, characterized in that: include: An image acquisition module, used to acquire a face image when a ringing event occurs; An image recognition module, used to recognize the face image and obtain user identity information and sight line recognition information; The volume adjustment module is used to adjust the ringing volume according to the user identity information and the sight line recognition information.

16. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 14 by executing the executable instructions.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

Images