User interaction method and device, electronic equipment, medium and program product

Abstract

The invention provides a user interaction method which can be applied to the technical field of financial science and technology. The method comprises the following steps: collecting and detecting sound signals around a user; in response to the detected target sound information, the target sound information is converted into a sound intensity signal, the target sound information comprises active expiration sound of a user or target environment sound, and the target environment sound is expressed as environment sound exceeding a noise level threshold value; generating a shielding special effect image based on the sound intensity signal, and performing real-time shielding on target sensitive information in an information display page by using the shielding special effect image; and in response to a target interaction operation of the user, gradually clearing the real-time shielding effect of the shielding special effect image, and displaying the shielded target sensitive information. The invention further provides a user interaction device, equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of financial technology, and more specifically to a user interaction method, apparatus, device, medium, and program product. Background Art

[0002] Existing interactive methods for displaying and hiding monetary data primarily rely on user-friendly button clicks, resulting in a relatively simple interaction format and a limited user experience. Traditional interaction methods are limited to displaying and hiding via physical or virtual buttons, lacking diverse and intelligent interactive methods, and failing to meet modern users' demand for natural interaction with smart devices. In widely used scenarios such as smartphones and tablets, users desire more intuitive and natural ways to protect their privacy and control sensitive data.

[0003] In addition, the traditional button operation mode usually requires frequent interface refreshes, especially during high-frequency operations, which consumes high system resources and increases response delays.

[0004] At present, some technical research attempts to improve the information display effect through complex occlusion effects or animation effects. Existing technologies cannot dynamically adjust the display and hiding of interface elements according to environmental changes. This requires the system to continuously perform global judgment and re-rendering, which increases the burden on the CPU and GPU and reduces overall performance. Summary of the Invention

[0005] In view of the above problems, the present disclosure provides a user interaction method, apparatus, device, medium and program product for optimizing the performance of dynamic special effects generation and improving user experience.

[0006] According to a first aspect of the present disclosure, a user interaction method is provided, comprising: collecting and detecting sound signals around a user; in response to detecting target sound information, converting the target sound information into a sound intensity signal; generating an occlusion special effect image based on the sound intensity signal, and utilizing the occlusion special effect image to perform real-time occlusion of target sensitive information in an information display page; and in response to a user's target interaction operation, clearing the real-time occlusion effect of the occlusion special effect image and displaying the occluded target sensitive information.

[0007] According to an embodiment of the present disclosure, after collecting and detecting the sound signals around the user, the method further includes: preprocessing the sound signals to extract sound signal features, wherein the sound signal features include frequency features, time features, and intensity features; and analyzing the sound signal features through a machine learning classification model to determine the type of the sound signal.

[0008] According to an embodiment of the present disclosure, the target sound information includes the user's active exhalation sound or the target environmental sound, and the target environmental sound is represented as an environmental sound exceeding a noise level threshold.

[0009] According to an embodiment of the present disclosure, the method also includes: in response to the type of the sound signal being the user's active exhalation sound, obtaining first distance sensor data, performing exhalation sound judgment based on the first distance sensor data, and obtaining a first judgment result; or in response to the type of the sound signal being the target ambient sound, obtaining first sound direction information of multiple microphone arrays, performing target ambient sound judgment based on the first sound direction information, and obtaining a second judgment result; and judging whether the target sound information is detected based on the first judgment result or the second judgment result.

[0010] According to an embodiment of the present disclosure, the target sound information also includes a user-specific sound signal, and the user-specific sound signal includes a clapping sound. The method also includes: in response to the type of the sound signal being a user-active sound signal, obtaining second distance sensor data and second sound direction information of a plurality of microphone arrays, performing sound signal judgment based on the second distance sensor data and the second sound direction information, and obtaining a third judgment result; and based on the third judgment result, judging whether the target sound information is detected.

[0011] According to an embodiment of the present disclosure, the occlusion special effects image includes a fog effect image, and generating the occlusion special effects image based on the sound intensity signal specifically includes: selecting fog effect parameters based on the intensity of the sound intensity signal, the fog effect parameters including fog concentration, fog diffusion speed and fog range; and based on the fog effect parameters, using image blur processing to simulate the fog effect to generate the occlusion special effects image.

[0012] According to an embodiment of the present disclosure, the occlusion special effects image includes a lightning effect image, and generating the occlusion special effects image based on the sound intensity signal specifically includes: selecting lightning effect parameters based on the intensity of the sound intensity signal, the lightning effect parameters including lightning brightness, number of lightning branches and lightning duration; and based on the lightning effect parameters, generating image data and lightning path using random noise and threshold processing, and generating lightning effects using fractal geometry to generate the occlusion special effects image.

[0013] According to an embodiment of the present disclosure, the use of the occlusion special effects image to perform real-time occlusion of target sensitive information in the information display page specifically includes: obtaining the location information of the target sensitive information; and based on the location information, using a graphics processing tool to achieve real-time rendering of the target sensitive information by the fog effect image, including: adjusting the fog effect parameters in real time according to the change of the sound intensity signal, and updating the occlusion special effects image in real time based on the adjusted fog effect parameters.

[0014] According to an embodiment of the present disclosure, the target interaction operation includes a gesture sliding operation or a shaking device operation, and the real-time occlusion effect of the occlusion special effect image is gradually cleared in response to the user's target interaction operation, and the occluded target sensitive information is displayed, specifically including: responding to the user's gesture sliding operation, gradually reducing the fog effect concentration of the fog effect image based on the sliding trajectory of the gesture sliding operation until the target sensitive information is fully revealed; or responding to the user's shaking device operation, reducing the fog effect concentration of the fog effect image based on the shaking amplitude and frequency of the shaking device operation until the target sensitive information is fully revealed.

[0015] According to an embodiment of the present disclosure, based on the size of the sound intensity signal, different target sensitive information is selected for real-time masking according to preset display rules.

[0016] A second aspect of the present disclosure provides a user interaction device, comprising: an acquisition module, configured to acquire and detect sound signals around a user; a sound intensity signal conversion module, configured to convert the target sound information into a sound intensity signal in response to detecting target sound information, wherein the target sound information includes the user's active exhalation sound or the target environmental sound, and the target environmental sound is represented as an environmental sound exceeding a noise level threshold; a blocking module, configured to generate a blocking special effect image based on the sound intensity signal, and utilize the blocking special effect image to perform real-time blocking of target sensitive information in an information display page; and a display module, configured to clear the real-time blocking effect of the blocking special effect image in response to the user's target interaction operation, and display the blocked target sensitive information.

[0017] According to an embodiment of the present disclosure, the acquisition module may include a special effect category and sound signal feature extraction unit and a sound signal type judgment unit.

[0018] According to an embodiment of the present disclosure, the sound signal feature extraction unit can be used to pre-process the sound signal to extract sound signal features.

[0019] According to an embodiment of the present disclosure, the sound signal type determination unit may be configured to analyze the sound signal features using a pre-trained machine learning classification model to determine the type of the sound signal.

[0020] According to an embodiment of the present disclosure, the acquisition module can also be used to obtain first distance sensor data in response to the type of sound signal being the user's active exhalation sound, perform exhalation sound judgment based on the first distance sensor data, and obtain a first judgment result.

[0021] According to an embodiment of the present disclosure, the acquisition module can also be used to obtain first sound direction information of multiple microphone arrays in response to the type of the sound signal being the target ambient sound, perform target ambient sound judgment based on the first sound direction information, and obtain a second judgment result.

[0022] According to an embodiment of the present disclosure, the acquisition module may also be configured to, in response to the sound signal being a user-initiated sound signal, obtain second distance sensor data and second sound direction information from the plurality of microphone arrays, perform sound signal determination based on the second distance sensor data and the second sound direction information, and obtain a third determination result. Based on the first, second, or third determination results, a determination may be made as to whether the target sound information is detected.

[0023] According to an embodiment of the present disclosure, the occlusion module may include a position information acquisition unit and a real-time rendering unit.

[0024] According to an embodiment of the present disclosure, the position information acquisition unit can be used to acquire the position information of the target sensitive information, including its position coordinates, size and boundary area on the screen.

[0025] According to an embodiment of the present disclosure, the real-time rendering unit can be used to realize real-time rendering of the target sensitive information by the fog effect image based on the position information using a graphics processing tool, including adjusting the fog effect parameters in real time according to changes in the sound intensity signal, and updating the occlusion special effects image in real time based on the adjusted fog effect parameters.

[0026] According to an embodiment of the present disclosure, the occlusion module can also be used to select atomization effect parameters based on the intensity of the sound intensity signal, and the atomization effect parameters include atomization concentration, atomization diffusion speed and atomization range; and based on the atomization effect parameters, use image blur processing to simulate the atomization effect to generate an occlusion special effects image.

[0027] According to an embodiment of the present disclosure, the occlusion module can also be used to select lightning effect parameters based on the intensity of the sound intensity signal, and the lightning effect parameters include lightning brightness, number of lightning branches and lightning duration; and based on the lightning effect parameters, generate image data and lightning paths using random noise and threshold processing, and generate lightning effects using fractal geometry to generate an occlusion special effects image.

[0028] According to an embodiment of the present disclosure, the shielding module may also be used to select different target sensitive information for real-time shielding according to preset display rules based on the size of the sound intensity signal.

[0029] According to an embodiment of the present disclosure, the shielding module may also be used to select different target sensitive information for real-time shielding according to preset display rules based on the size of the sound intensity signal.

[0030] According to an embodiment of the present disclosure, the display module can also be used to respond to the user's gesture sliding operation, and gradually reduce the fog effect concentration of the fog effect image based on the sliding trajectory of the gesture sliding operation until the target sensitive information is fully revealed.

[0031] According to an embodiment of the present disclosure, the display module can also be used to respond to the user's shaking device operation, and reduce the fog effect concentration of the fog effect image based on the shaking amplitude and frequency of the shaking device operation until the target sensitive information is fully revealed.

[0032] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0033] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.

[0034] The fifth aspect of the present disclosure further provides a computer program product, comprising a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0035] According to the embodiments of the present disclosure, dynamic masking and revealing of sensitive monetary information is achieved by detecting special sound signals around the user and combining them with gesture sliding or device shaking. This method breaks away from the traditional single button operation mode and utilizes the user's natural behavior, such as exhalation, to interact, making the display and hiding of private information more in line with people's intuition and usage habits, greatly improving the user's interactive experience and the flexibility of privacy protection. It also avoids the system's long-term blocking of all sensitive information, thereby reducing system resource usage and improving timely responsiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0037] Figure 1Schematically illustrates an application scenario diagram of the user interaction method, apparatus, device, medium, and program product according to an embodiment of the present disclosure;

[0038] Figure 2 The following schematically shows a flow chart of a user interaction method according to an embodiment of the present disclosure;

[0039] Figure 3 The following schematically illustrates a method for determining the type of a sound signal according to an embodiment of the present disclosure;

[0040] Figure 4 Schematically illustrates a method for implementing occlusion using an occlusion special effect image according to an embodiment of the present disclosure;

[0041] Figure 5 A structural block diagram of a user interaction device according to an embodiment of the present disclosure is schematically shown; and

[0042] Figure 6 A block diagram of an electronic device suitable for implementing a user interaction method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0043] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0044] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0045] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0046] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0047] The existing interactive method for displaying and hiding amount data mainly relies on users clicking buttons. The interactive form is relatively simple and the user experience is relatively limited. At the same time, the existing interactive system also has significant limitations in computer performance, which are mainly reflected in the following points:

[0048] 1. High resource usage: Traditional button operation modes usually require frequent interface refreshes, especially during high-frequency operations, which consumes high system resources and increases response delays.

[0049] 2. Lack of real-time dynamic processing capabilities: Existing technologies cannot dynamically adjust the display and hiding of interface elements based on environmental changes. This requires the system to continuously perform global judgments and re-render, increasing the burden on the CPU and GPU and reducing overall performance.

[0050] 3. Inefficiency of the interaction process: A single button operation method cannot achieve intelligent interactive control. The system's perception and processing efficiency of user operations is low, and it cannot flexibly respond to users' natural behaviors (such as exhalation and gestures), resulting in waste of resources and inefficient interaction.

[0051] In addition, this traditional interaction method also has certain bottlenecks in computer performance, especially in multi-tasking scenarios. Frequent button operations may cause excessive system resource usage, reducing the overall response speed and the smoothness of the user experience.

[0052] Based on this, embodiments of the present disclosure provide a user interaction method, comprising: collecting and detecting sound signals around a user; in response to detecting target sound information, converting the target sound information into a sound intensity signal, wherein the target sound information includes the user's active exhalation sound or target ambient sound, wherein the target ambient sound is defined as ambient sound exceeding a noise level threshold; generating a masking effect image based on the sound intensity signal, and utilizing the masking effect image to mask target sensitive information on an information display page in real time; and, in response to the user's target interaction operation, gradually removing the real-time masking effect of the masking effect image to display the masked target sensitive information. The user interaction method provided by the present disclosure dynamically masks and reveals sensitive monetary information by detecting specific sound information around the user and combining it with gestures such as sliding or shaking the device. This method breaks away from the traditional single-button operation mode and utilizes the user's natural behaviors, such as exhalation and gestures, to interact, making the display and concealment of private information more intuitive and user-friendly, greatly enhancing the user interaction experience and the flexibility of privacy protection.

[0053] It should be noted that the user interaction methods, devices, equipment, media, and program products identified in this disclosure can be used in the field of financial technology, and can also be used in various fields outside of the field of financial technology. The application fields of the user interaction methods, devices, equipment, media, and program products provided in the embodiments of this disclosure are not limited.

[0054] In the technical solutions disclosed herein, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0055] In scenarios where personal information is used for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure all provide users with corresponding operation portals for them to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge, and skills, and have reached a certain level of professionalism.

[0056] Figure 1 The application scenario diagram of the user interaction method, apparatus, device, medium and program product according to the embodiments of the present disclosure is schematically shown.

[0057] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or optical fiber cables.

[0058] A user may use a first terminal device 101, a second terminal device 102, or a third terminal device 103 to interact with a server 105 via a network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, or the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0059] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0060] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal devices.

[0061] It should be noted that the user interaction method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the user interaction device provided in the embodiment of the present disclosure can generally be set in the server 105. The user interaction method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the user interaction device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0062] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0063] The following will be based on Figure 1 The scene described by Figures 2 to 4 The user interaction method of the disclosed embodiment is described in detail.

[0064] Figure 2 The flowchart of the user interaction method according to the embodiment of the present disclosure is schematically shown.

[0065] like Figure 2 As shown, the user interaction method of this embodiment includes operations S210 to S240, and the transaction processing method can be executed by the server 105.

[0066] In operation S210 , a sound signal around a user is collected and detected.

[0067] In embodiments of the present disclosure, a microphone can be used to capture real-time sounds around the device, including active user vocalizations (such as blowing or clapping) and background noise. When a user actively blows into the microphone at close range, or when a noisy environment is detected, the embodiment of the present disclosure can extract the size of the sound waves, control the degree of hiding of sensitive information, and present it in a style that simulates a specific occlusion effect.

[0068] It should be noted that the user interaction methods, devices, equipment, media, and program products provided by the embodiments of the present disclosure serve as personalized interaction methods, giving users the initiative to protect their privacy. That is, users can choose to turn the occlusion special effects function on or off in the settings menu, freely choosing whether to use the occlusion special effects. Before using this function for the first time, the user's consent or authorization may be obtained. For example, before operation S210, an authorization request may be issued to the user. If the user agrees or authorizes use, operation S210 is performed.

[0069] In order to implement different processing methods for different sound signals and achieve flexibility in interaction and presentation, the embodiments of the present disclosure may first determine the type of the sound signal after collecting the sound signal.

[0070] Figure 3 The following schematically illustrates a method for determining the type of a sound signal according to an embodiment of the present disclosure.

[0071] like Figure 3 As shown, the method for determining the type of the sound signal in this embodiment may include operations S310 to S320.

[0072] In operation S310, the sound signal is preprocessed to extract sound signal features, wherein the sound signal features include frequency features, time features, and intensity features.

[0073] In the embodiments of the present disclosure, the collected sound signals can be processed by digital signals (such as filtering, denoising, etc.) to extract key information, such as frequency, intensity, duration and other features, for further judgment and classification.

[0074] For example, regarding frequency characteristics, exhalation sounds typically have continuous, low-frequency sound waves within a relatively concentrated frequency range, while ambient noise is typically distributed over a wider frequency range with irregular fluctuations. Specifically, frequency domain analysis methods such as Fast Fourier Transform (FFT) can be used to extract the frequency components of the sound signal. If the signal frequency is concentrated within a specific, low range, it can be identified as exhalation.

[0075] Regarding temporal characteristics, exhalation sounds have a short duration and relatively concentrated signal energy, appearing as a one-time, continuous waveform. Environmental noise, on the other hand, typically exhibits a longer, more variable waveform, especially in public places, where noise can exhibit significant frequency and intensity variations. Specifically, the sound signal's short-term energy and duration can be analyzed to identify it. If the duration is short and the energy curve is relatively smooth, it can be identified as an exhalation.

[0076] Regarding intensity, the intensity of exhalation sounds typically rises and then falls, forming a typical "arch" waveform. However, noise intensity in noisy environments often exhibits irregular fluctuations. Signal envelope extraction techniques can be used to analyze the intensity curve of a sound and determine whether it is an exhalation sound.

[0077] In operation S320, the sound signal features are analyzed using a pre-trained machine learning classification model to determine the type of the sound signal.

[0078] Furthermore, in order to more accurately identify the user's intention and sound source and ensure that the system can correctly respond to the user's operation, further analysis can be performed by combining multiple sensor data.

[0079] In an embodiment of the present disclosure, in response to the sound signal being a user's active exhalation, first distance sensor data is acquired, and an exhalation sound determination is performed based on the first distance sensor data to obtain a first determination result. Exhalation sounds are a typical example of active user interaction, but they may be confused with other sounds in an environment. By acquiring distance sensor data, it is possible to determine whether the user is approaching the device, thereby confirming whether the sound is indeed an active user interaction.

[0080] In an embodiment of the present disclosure, in response to the type of sound signal being a target ambient sound, first sound direction information of a plurality of microphone arrays is obtained, and a target ambient sound judgment is performed based on the first sound direction information to obtain a second judgment result. The target ambient sound represents the user's environmental state and can be represented as ambient sound that exceeds a noise level threshold. When the noise level exceeds this threshold, it can be considered that the environment is noisy enough and certain measures need to be taken to shield sensitive information, thereby improving the effectiveness of user privacy protection. Therefore, determining the source direction of these ambient sounds can help determine whether the sound is a type of sound that the system needs to pay attention to.

[0081] Specifically, the noise level threshold can be adjusted according to specific application scenarios. For example, 65 dB is usually suitable for daily life environments. If the background noise level continuously exceeds 65 dB, it can be considered a noisy environment.

[0082] In an embodiment of the present disclosure, in response to the sound signal being a user-initiated sound signal, second distance sensor data and second sound direction information from a plurality of microphone arrays are obtained, and a sound signal determination is performed based on the second distance sensor data and the second sound direction information to obtain a third determination result. User-specific sound signals include clapping, coughing, laughter, and the like, thereby providing a diverse interactive experience.

[0083] For example, when detecting clapping sounds, the proximity sensor and multiple microphone arrays can be used to confirm whether the user is within the device's operating range. If the clapping sound occurs when the user is close to the device and comes from the user's direction, it is more likely to be an active user interaction.

[0084] According to the embodiments of the present disclosure, based on the first, second, or third judgment results, it is possible to further determine whether the target sound information is detected. By combining information from the distance sensor and the microphone array, the system can more accurately determine whether a specific sound actively emitted by the user is an interaction signal, avoiding misjudgment of irrelevant sounds.

[0085] Return to reference Figure 2 In operation S220, in response to detecting target sound information, the target sound information is converted into a sound intensity signal, wherein the target sound information includes the user's active exhalation sound or the target environmental sound.

[0086] In embodiments of the present disclosure, target sound information can be converted into a sound intensity signal by calculating the amplitude of the sound waveform per unit time. Specifically, the peak or average value of the sound signal can be converted into a specific intensity value to quantify the strength of the sound. The purpose of converting the target sound information into a sound intensity signal is to provide a standardized input for the subsequent generation of occlusion special effect images, allowing the sound intensity signal to be easily used to adjust the strength of the occlusion effect.

[0087] Preferably, the intensity of the sound signal needs to be calculated in real time, meaning that the sound intensity signal is a dynamic quantity, allowing the system to generate corresponding occlusion effects based on the dynamic changes in the sound signal. Real-time intensity detection ensures that the system can immediately make corresponding visual occlusion adjustments as the user's sound input changes. By real-time detection of sound signal intensity, the complexity of the occlusion effects can also be controlled, ensuring visual quality while avoiding excessive consumption of computing resources and maintaining system responsiveness.

[0088] In operation S230, a blocking special effect image is generated based on the sound intensity signal, and the target sensitive information in the information display page is blocked in real time using the blocking special effect image.

[0089] In an embodiment of the present disclosure, if the target sound information is the user's active exhalation sound or the target ambient sound, the occlusion special effect image may be a fog effect image. Generating the occlusion special effect image may specifically include: selecting fog effect parameters based on the strength of the sound intensity signal, the fog effect parameters including fog concentration, fog diffusion speed, and fog range; and simulating the fog effect based on the fog effect parameters and image blur processing to generate the occlusion special effect image.

[0090] Specifically, image processing algorithms such as Gaussian blur can be used to blur images in areas where sensitive information is located. The concentration of the fog determines the degree of blur, while the rate of diffusion determines the duration of the blur effect. The fog effect simulates a layer of mist forming on glass, making sensitive information difficult to discern. By adjusting the fog effect parameters in real time, the fogged image can dynamically change with changes in the user's voice signal, thereby responding to user interactions in real time.

[0091] In an embodiment of the present disclosure, corresponding to the target sound information being a user-specific sound signal, the occlusion special effect image can be generated as a lightning effect image with stronger visual impact, which is used to occlude sensitive information areas.

[0092] Specifically, lightning effect parameters can be selected based on the strength of the sound intensity signal, including lightning brightness, number of lightning branches, and lightning duration. Based on these lightning effect parameters, random noise and threshold processing are used to generate image data and lightning paths, and fractal geometry is used to generate lightning effects, thereby generating an occlusion special effect image. Similarly, the dynamic generation process of the lightning effect can also be adjusted based on the user's sound intensity signal to align with the user's voice interaction.

[0093] Figure 4 The method for implementing occlusion using an occlusion special effect image according to an embodiment of the present disclosure is schematically illustrated.

[0094] like Figure 4 As shown, the method for implementing occlusion using an occlusion special effect image in this embodiment may include operations S410 to S420.

[0095] In operation S410 , location information of target sensitive information is acquired, including its location coordinates, size, and boundary area on the screen.

[0096] In operation S420, based on the position information, a graphics processing tool is used to implement real-time rendering of the target sensitive information by the fog effect image, including adjusting the fog effect parameters in real time according to the change of the sound intensity signal, and updating the occlusion special effect image in real time based on the adjusted fog effect parameters.

[0097] For example, as sound intensity increases, the atomization density increases, making sensitive information more obscured and difficult to discern. Conversely, as sound intensity decreases, the atomization density decreases. Greater sound intensity leads to faster atomization, enabling complete obstruction in a shorter period of time. For signals with higher sound intensity, a larger obstruction range can be selected to ensure that sensitive information is completely covered.

[0098] In the embodiments of the present disclosure, the system can pre-define a set of display rules to determine which sensitive information needs to be blocked at different sound intensities. Based on the magnitude of the sound intensity signal, different target sensitive information can be selected for real-time blocking according to the preset display rules. For example, when the sound intensity is high, income information can be selected for blocking, while when the sound intensity is low, expenditure information can be selected for blocking, and so on.

[0099] Return to reference Figure 2 In operation S240, in response to the user's target interaction operation, the real-time occlusion effect of the occlusion special effect image is gradually cleared, and the occluded target sensitive information is displayed.

[0100] In an embodiment of the present disclosure, the target interaction operation includes a gesture sliding operation or a device shaking operation.

[0101] In an embodiment of the present disclosure, in response to a user's sliding gesture, the concentration of the fog effect of the fogged image can be gradually reduced based on the sliding trajectory of the gesture sliding operation until the target sensitive information is fully revealed. Specifically, the touch sensor can capture the sliding position and trajectory of the user's finger on the screen to determine the path and range of the fog effect removal. By gradually reducing the fog concentration in this way, the user can gradually reveal the target sensitive information instead of revealing all the information at once, thereby increasing the controllability of information disclosure and enhancing user participation and interactive fun.

[0102] In an embodiment of the present disclosure, in response to the user's operation of shaking the device, the concentration of the atomization effect of the atomization effect image can be reduced based on the shaking amplitude and frequency of the shaking device operation until the target sensitive information is fully revealed. Specifically, the built-in acceleration sensor can be used to detect the shaking amplitude and frequency of the device to simulate the effect of wind. When a shaking action is detected, the system calculates the motion parameters of the device in three-dimensional space. When the shaking amplitude is large, the system will reduce the concentration of the atomization effect more quickly, and when the shaking amplitude is small, the concentration will decrease more slowly. Among them, the relationship between the amplitude and the response can provide the user with certain interactive feedback, so that the strength of the shaking operation is proportional to the speed of information disclosure.

[0103] The user interaction method proposed in the embodiment of the present disclosure can have the following beneficial effects:

[0104] 1. Allow users to interact with the system through natural actions (such as exhaling, swiping gestures, and shaking the device), replacing traditional button clicks. This contactless interaction method that conforms to human nature significantly improves the user experience and enables users to control the display and hiding of information in a more intuitive and convenient way.

[0105] 2. By dynamically adjusting occlusion effects (such as fog density and range) in real time, users can see information gradually become clearer from blurry. This gradual visual effect provides an excellent user experience, allowing users to better control the degree of information revealed during interaction, thereby reducing concerns about privacy leaks and increasing the fun of interaction.

[0106] 3. This method uses real-time, intelligent dynamic processing to generate and adjust occlusion effects based on sound intensity signals and user interaction. This strategy of dynamically generating images based on user input reduces unnecessary occlusion operations and prevents the system from permanently blocking all sensitive information, thereby reducing system resource usage, especially in image rendering.

[0107] Based on the above user interaction method, the present disclosure also provides a user interaction device. Figure 5 The device is described in detail.

[0108] Figure 5 The structural block diagram of the user interaction device according to an embodiment of the present disclosure is schematically shown.

[0109] like Figure 5 As shown, the user interaction device 500 of this embodiment includes a collection module 510 , a sound intensity signal conversion module 520 , a shielding module 530 , and a display module 540 .

[0110] The acquisition module 510 may be used to acquire and detect sound signals around the user. In one embodiment, the acquisition module 510 may be used to perform the operation S210 described above, which will not be described in detail here.

[0111] The sound intensity signal conversion module 520 can be configured to, in response to detecting target sound information, convert the target sound information into a sound intensity signal. The target sound information includes the user's active exhalation sound or target ambient sound, where the target ambient sound is defined as ambient sound exceeding a noise level threshold. In one embodiment, the sound intensity signal conversion module 520 can be configured to perform operation S220 described above, and will not be further described here.

[0112] The shielding module 530 can be used to generate a shielding special effect image based on the sound intensity signal, and use the shielding special effect image to shield the target sensitive information in the information display page in real time. In one embodiment, the shielding module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0113] The display module 540 can be used to respond to the user's target interaction operation, gradually remove the real-time occlusion effect of the occlusion special effect image, and display the occluded target sensitive information. In one embodiment, the display module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0114] According to an embodiment of the present disclosure, the acquisition module 510 may include a special effect category and sound signal feature extraction unit and a sound signal type determination unit.

[0115] The sound signal feature extraction unit may be used to pre-process the sound signal and extract sound signal features, wherein the sound signal features include frequency features, time features, and intensity features. In one embodiment, the sound signal feature extraction unit may be used to perform operation S310 described above, which will not be repeated here.

[0116] The sound signal type determination unit may be configured to analyze the sound signal features using a pre-trained machine learning classification model to determine the type of the sound signal. In one embodiment, the sound signal type determination unit may be configured to perform operation S320 described above, which will not be described in detail herein.

[0117] According to an embodiment of the present disclosure, the acquisition module can also be used to obtain first distance sensor data in response to the type of sound signal being the user's active exhalation sound, perform exhalation sound judgment based on the first distance sensor data, and obtain a first judgment result.

[0118] According to an embodiment of the present disclosure, the acquisition module can also be used to obtain first sound direction information of multiple microphone arrays in response to the type of the sound signal being the target ambient sound, perform target ambient sound judgment based on the first sound direction information, and obtain a second judgment result.

[0119] According to an embodiment of the present disclosure, the acquisition module may also be configured to, in response to the sound signal being a user-initiated sound signal, obtain second distance sensor data and second sound direction information from the plurality of microphone arrays, perform sound signal determination based on the second distance sensor data and the second sound direction information, and obtain a third determination result. Based on the first, second, or third determination results, a determination may be made as to whether the target sound information is detected.

[0120] According to an embodiment of the present disclosure, the blocking module 530 may include a position information acquiring unit and a real-time rendering unit.

[0121] The position information acquisition unit can be used to obtain the position information of the target sensitive information, including its position coordinates, size and boundary area on the screen. In one embodiment, the position information acquisition unit can be used to perform the operation S410 described above, which will not be repeated here.

[0122] The real-time rendering unit can be configured to use a graphics processing tool to implement real-time rendering of the target sensitive information using the fog effect image based on the location information, including adjusting fog effect parameters in real time according to changes in the sound intensity signal, and updating the occlusion special effect image in real time based on the adjusted fog effect parameters. In one embodiment, the real-time rendering unit can be configured to perform operation S420 described above, which will not be further described here.

[0123] According to an embodiment of the present disclosure, the occlusion module can also be used to select atomization effect parameters based on the intensity of the sound intensity signal, and the atomization effect parameters include atomization concentration, atomization diffusion speed and atomization range; and simulate the atomization effect based on the atomization effect parameters and image blur processing to generate an occlusion special effects image.

[0124] According to an embodiment of the present disclosure, the occlusion module can also be used to select lightning effect parameters based on the intensity of the sound intensity signal, and the lightning effect parameters include lightning brightness, number of lightning branches and lightning duration; and based on the lightning effect parameters, generate image data and lightning paths using random noise and threshold processing, and generate lightning effects using fractal geometry to generate an occlusion special effects image.

[0125] According to an embodiment of the present disclosure, the shielding module may also be used to select different target sensitive information for real-time shielding according to preset display rules based on the size of the sound intensity signal.

[0126] According to an embodiment of the present disclosure, the shielding module may also be used to select different target sensitive information for real-time shielding according to preset display rules based on the size of the sound intensity signal.

[0127] According to an embodiment of the present disclosure, the display module can also be used to respond to the user's gesture sliding operation, and gradually reduce the fog effect concentration of the fog effect image based on the sliding trajectory of the gesture sliding operation until the target sensitive information is fully revealed.

[0128] According to an embodiment of the present disclosure, the display module can also be used to respond to the user's shaking device operation, and reduce the fog effect concentration of the fog effect image based on the shaking amplitude and frequency of the shaking device operation until the target sensitive information is fully revealed.

[0129] According to embodiments of the present disclosure, any multiple modules among the information acquisition module 510, special effect image generation module 520, dynamic occlusion module 530, occlusion removal module 540, and re-occlusion module 550 may be combined into a single module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in a single module. According to embodiments of the present disclosure, at least one of the information acquisition module 510, special effect image generation module 520, dynamic occlusion module 530, occlusion removal module 540, and re-occlusion module 550 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented in hardware or firmware by any other reasonable means of circuit integration or packaging, or may be implemented in any one of the three implementation methods of software, hardware, and firmware, or any appropriate combination of any of these. Alternatively, at least one of the information acquisition module 510, the special effect image generation module 520, the dynamic occlusion module 530, the occlusion release module 540 and the re-occlusion module 550 can be at least partially implemented as a computer program module, which can perform corresponding functions when executed.

[0130] Figure 6 A block diagram of an electronic device suitable for implementing a user interaction method according to an embodiment of the present disclosure is schematically shown.

[0131] like Figure 6 As shown, an electronic device 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0132] Various programs and data required for the operation of the electronic device 600 are stored in the RAM 603. The processor 601, ROM 602, and RAM 603 are connected to each other via a bus 604. The processor 601 executes the various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than the ROM 602 and RAM 603. The processor 601 may also execute the various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0133] According to an embodiment of the present disclosure, electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to bus 604. Electronic device 600 may also include one or more of the following components connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN card or modem. Communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. Removable media 611, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 610 as needed, so that computer programs read from the removable media can be installed into storage section 608 as needed.

[0134] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0135] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 602 and / or RAM 603 described above, and / or one or more memories other than ROM 602 and RAM 603.

[0136] The embodiments of the present disclosure also include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the user interaction method provided by the embodiments of the present disclosure.

[0137] The computer program executes the above functions defined in the system / device of the embodiment of the present disclosure when the processor 601 executes the computer program. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0138] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0139] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from a removable medium 611. When the computer program is executed by the processor 601, the above-described functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0140] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] Those skilled in the art will appreciate that the features described in the various embodiments of the present disclosure may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in the present disclosure. In particular, the features described in the various embodiments of the present disclosure may be combined and / or coupled in various ways without departing from the spirit and teachings of the present disclosure. All such combinations and / or couplings fall within the scope of the present disclosure.

[0143] The above describes the embodiments of the present disclosure. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A user interaction method, characterized in that: The method comprises: Collect and detect sound signals around the user; In response to detecting target sound information, converting the target sound information into a sound intensity signal; generating a masking special effect image based on the sound intensity signal, and using the masking special effect image to mask target sensitive information in the information display page in real time; and In response to the user's target interaction operation, the real-time occlusion effect of the occlusion special effect image is cleared, and the occluded target sensitive information is displayed.

2. The method according to claim 1, characterized in that After collecting and detecting the sound signals around the user, the method further includes: Preprocessing the sound signal to extract sound signal features, wherein the sound signal features include frequency features, time features, and intensity features; and The sound signal characteristics are analyzed through a machine learning classification model to determine the type of the sound signal.

3. The method according to claim 2, characterized in that The target sound information includes the user's active exhalation sound or a target environmental sound, where the target environmental sound is an environmental sound exceeding a noise level threshold.

4. The method according to claim 2, characterized in that The method further comprises: In response to the sound signal being an active exhalation sound of the user, obtaining first distance sensor data, performing an exhalation sound judgment based on the first distance sensor data, and obtaining a first judgment result; and determining whether the target sound information is detected based on the first judgment result; or In response to the type of the sound signal being the target ambient sound, first sound direction information of multiple microphone arrays is obtained, the target ambient sound is judged based on the first sound direction information, and a second judgment result is obtained; based on the second judgment result, it is determined whether the target sound information is detected.

5. The method according to claim 2, characterized in that The target sound information further includes a user-specific sound signal, and the user-specific sound signal includes a clapping sound. The method further includes: In response to the sound signal being a user-initiated sound signal, obtaining second distance sensor data and second sound direction information of the plurality of microphone arrays, performing sound signal judgment based on the second distance sensor data and the second sound direction information, and obtaining a third judgment result; and Based on the third judgment result, it is determined whether the target sound information is detected.

6. The method according to any one of claims 1 to 4, characterized in that The occlusion special effect image includes a fog effect image, and generating the occlusion special effect image based on the sound intensity signal specifically includes: selecting atomization effect parameters based on the intensity of the sound intensity signal, the atomization effect parameters including atomization concentration, atomization diffusion speed, and atomization range; and Based on the fog effect parameters, the fog effect is simulated by using image blur processing to generate an occlusion special effect image.

7. The method according to claim 5, characterized in that The occlusion special effect image includes a lightning effect image, and generating the occlusion special effect image based on the sound intensity signal specifically includes: selecting lightning effect parameters based on the intensity of the user-specific sound signal, the lightning effect parameters including lightning brightness, number of lightning branches, and lightning duration; and Based on the lightning effect parameters, random noise and threshold processing are used to generate image data and lightning paths, and fractal geometry is used to generate lightning effects, so as to generate an occlusion special effect image.

8. The method according to claim 6, characterized in that The using of the masking special effect image to mask the target sensitive information in the information display page in real time specifically includes: Obtaining the location of sensitive target information; and Based on the position information, a graphics processing tool is used to realize real-time rendering of the target sensitive information by the fog effect image, including adjusting the fog effect parameters in real time according to the change of the sound intensity signal, and updating the occlusion special effect image in real time based on the adjusted fog effect parameters.

9. The method according to claim 6, characterized in that The target interaction operation includes a gesture sliding operation or a device shaking operation. In response to the user's target interaction operation, clearing the real-time occlusion effect of the occlusion special effect image and displaying the occluded target sensitive information specifically includes: In response to a user's sliding gesture operation, gradually reducing the concentration of the atomized effect of the atomized effect image based on the sliding trajectory of the sliding gesture operation until the target sensitive information is completely revealed; or In response to a user's shaking operation of the device, the fog effect density of the fog effect image is reduced based on the shaking amplitude and frequency of the shaking operation of the device until the target sensitive information is completely revealed.

10. The method according to claim 6, characterized in that The method further comprises: Based on the size of the sound intensity signal, different target sensitive information is selected for real-time masking according to preset display rules.

11. A user interaction device, characterized in that: The device comprises: The acquisition module is used to collect and detect the sound signals around the user; a sound intensity signal conversion module, configured to: in response to detecting target sound information, convert the target sound information into a sound intensity signal; A shielding module is configured to generate a shielding special effect image based on the sound intensity signal, and use the shielding special effect image to shield target sensitive information in the information display page in real time; and The display module is used to: in response to the user's target interaction operation, clear the real-time occlusion effect of the occlusion special effect image and display the occluded target sensitive information.

12. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images