Sound effect control method and device and computing equipment

Abstract

The embodiment of the invention provides a sound effect control method and device and computing equipment. The sound effect control method comprises the steps that a scene picture to be controlled is divided into at least two scene areas; in response to a sound effect control event of a target scene region, determining region content information of the target scene region, and determining a target sound effect parameter of the target scene region according to the region content information, the target scene region being any one of the at least two scene regions; and controlling the sound effect of the target scene area according to the target sound effect parameter of the target scene area. Regional division is carried out on a scene picture to be controlled, sound effects of different scene regions are independently regulated and controlled, different sound effects are allowed to be played in different scene regions, the layering sense and dynamic expressive force of the sound effects are improved, dynamic sound effect control is achieved based on region content information of scene dynamic change, and the user experience is improved. The sound effect in a complex scene is prevented from being disjointed with an actual scene, and the control effect of the sound effect is improved.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and more particularly to a sound effect control method, device, and computing equipment. Background Art

[0002] With the rapid development of computer technology and Internet technology, various multimedia scenes emerge in an endless stream and have become an indispensable entertainment for most people. In order to make the object have a more realistic experience, corresponding sound effects can be set in various multimedia scenes. When designers develop, in addition to designing and developing necessary scenes, characters, rules and other contents, another important development part is scene sound effects. Sound effect design is crucial to enhancing the immersion of the object.

[0003] In the prior art, when designing sound effects for multimedia scenes, fixed sound effects are usually played based on preset trigger conditions, which lacks dynamic response to the scene content, resulting in the sound effects being out of touch with the actual scene content in complex scenes, poor sound effect control, and thus poor immersion and experience of the subject. Therefore, there is an urgent need for a sound effect control solution that can enhance the immersion and experience of the subject. Summary of the invention

[0004] In view of this, an embodiment of this specification provides a sound effect control method. One or more embodiments of this specification also relate to a sound effect control device, a computing device, a computer readable storage medium and a computer program product to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a sound effect control method is provided, comprising: Divide the scene picture to be controlled into at least two scene areas; In response to a sound effect control event of a target scene area, determining area content information of the target scene area, and determining a target sound effect parameter of the target scene area according to the area content information, wherein the target scene area is any one of the at least two scene areas; The sound effect of the target scene area is controlled according to the target sound effect parameters of the target scene area.

[0006] According to a second aspect of an embodiment of this specification, there is provided a sound effect control device, comprising: A division module, configured to divide the scene picture to be controlled into at least two scene areas; a determination module configured to determine, in response to a sound effect control event of a target scene area, area content information of the target scene area, and determine a target sound effect parameter of the target scene area according to the area content information, wherein the target scene area is any one of the at least two scene areas; The control module is configured to control the sound effect of the target scene area according to the target sound effect parameters of the target scene area.

[0007] According to a third aspect of an embodiment of this specification, a computing device is provided, including: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned sound effect control method are implemented.

[0008] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned sound effect control method are implemented.

[0009] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instruction, which implements the steps of the above-mentioned sound effect control method when executed by a processor.

[0010] The embodiment of this specification provides a sound effect control method, which realizes dividing the scene picture to be controlled into at least two scene areas, and for any target scene area in the at least two scene areas, responding to the sound effect control event of the target scene area, determining the regional content information of the target scene area, and determining the target sound effect parameters of the target scene area based on the currently determined regional content information, so as to realize the sound effect control of the target scene area. In this way, by dividing the scene picture into regions, the sound effects of different scene areas can be independently regulated, and the changes in regional content information can be flexibly responded to, allowing different scene areas to play different sound effects, enriching the sound effect content, and improving the layering and dynamic expression of the sound effect; and for independent scene areas, by identifying the current regional content information to determine the corresponding target sound effect parameters, dynamic sound effect control can be realized based on the regional content information of the scene's dynamic changes, ensuring that the sound effect is synchronized with the content information changes of the scene area, avoiding the sound effect being out of touch with the actual scene content in complex scenes, improving the control effect of the sound effect, and thus improving the immersion and experience of the object. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a flow chart of a sound effect control method provided by an embodiment of this specification; Figure 2is a processing flow chart of a sound effect control method provided by an embodiment of this specification; Figure 3 It is a structural schematic diagram of a sound effect control device provided by an embodiment of this specification; Figure 4 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0012] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0013] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0014] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0015] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0016] First, the terms involved in one or more embodiments of this specification are explained.

[0017] HSV color space: is a color representation method based on human intuition, which divides color into three basic components: hue, saturation, and value. HSV color space is sometimes also called HSB (Hue, Saturation, Brightness) color space, but in most cases, they refer to the same concept. The HSV color space is designed to be closer to the way humans perceive color rather than the way electronic devices display it. Hue (Hue, H) refers to the basic type of color, such as red, green, blue, etc. In the HSV color space, hue is represented on the color wheel in the form of angles, ranging from 0° to 360°, 0° represents red, 120° represents green, 240° represents blue, and other angles on the color wheel correspond to other colors. Saturation (S) indicates the purity of a color, that is, the degree to which the color does not contain white light. A color with a saturation of 0% is a gray tone (including black and white). As the saturation increases, the color becomes more vivid. The saturation range is generally from 0% (completely desaturated, that is, gray) to 100% (completely saturated, that is, the brightest color). Value (V) Value indicates the brightness or darkness of a color. In the HSV color space, the value varies from 0 (completely dark, that is, black) to 1 (completely bright, that is, white). When the value of the value is 0, the color is black regardless of the hue and saturation; when the value of the value is 1, the color is the brightest, but the specific color also depends on the values ​​of the hue and saturation.

[0018] Edge detection: It is a basic technology in computer vision and image processing. It is used to identify the boundaries or contours of objects in images. Edge detection algorithms can help extract important features of images and are often used in tasks such as image segmentation, target recognition, and feature extraction.

[0019] Edge detection is based on the following assumptions: Brightness change: Edges usually appear where the brightness (grayscale value) in the image changes sharply; Gradient information: Edges can be regarded as local maximum points of the image gradient (i.e., the rate of change of brightness).

[0020] Optical Flow Estimation: It is a computer vision technology used to estimate the movement of pixels in an image sequence. Specifically, optical flow refers to the displacement of light points reflected from the surface of the same object on the image plane in two or more consecutive frames of images. By analyzing these displacements, the direction and speed of the object's movement can be inferred.

[0021] Optical flow detection is based on the following assumptions: constant brightness assumption: in a short period of time, the intensity of light reflected from the surface of an object remains unchanged, that is, the pixel brightness changes very little between adjacent frames; small motion assumption: between two frames, the movement of the object is small and can be approximated as linear motion; spatial consistency assumption: adjacent pixels usually have similar motion characteristics.

[0022] It should be noted that, taking the game scene as an example, in game development, sound effect design is crucial to enhancing the immersion of the game. How to make the sound effects dynamically adapt to the environmental changes of the game scene and enhance the immersion has become a key technical point. At present, the sound effects in the game scene are pre-designed and played according to fixed trigger conditions, that is, fixed sound effects are usually played based on preset trigger conditions, lacking dynamic response to real-time scenes and object behaviors, resulting in the sound effects being out of touch with the actual scenes in complex game scenes. With the rise of open world and multiplayer online games, the complexity of game scenes and the requirements for object interactivity are also increasing. Fixed sound effect triggering methods are difficult to cope with more dynamic and complex scene requirements. The sound effects in the game need to be adjusted in real time according to the complexity of the scene and changes in object interactions to enhance the overall immersion and game experience.

[0023] Therefore, the embodiments of this specification provide a sound effect control solution, which dynamically adjusts the sound effects by monitoring the real-time area content information in the scene image to be controlled, ensures that the sound effects are synchronized with scene changes and object behaviors, uses color, shape detection and motion detection technology to monitor the number and behavior of objects in real time, and dynamically adjusts the sound effect content based on the monitoring results, and combines with rule-driven sound effect control methods to provide smooth transition sound effect changes, which is more real-time and flexible in sound effect control.

[0024] In this specification, a sound effect control method is provided. This specification also relates to a sound effect control device, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0025] See also Figure 1 , Figure 1 A flow chart of a sound effect control method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0026] Step 102: Divide the scene image to be controlled into at least two scene areas.

[0027] The scene screen to be controlled is the scene screen that needs to be controlled by sound effects. For example, in a game scene, the scene screen to be controlled is the game screen; in a video scene, the scene screen to be controlled is the video screen; in an animation scene, the scene screen to be controlled is the animation screen.

[0028] In actual implementation, the area division rules of the scene screen to be controlled can be pre-defined, and the area division rules are used to divide the scene screen to be controlled into at least two scene areas according to the corresponding rule constraints. For example, the area division rules can be divided according to the defined area range, such as dividing the scene screen to be controlled into X areas; or, in the game scene, the area division can be performed according to the map blocks, such as the city map block, the suburban map block, the A scene map block, the B scene map block, etc.; or, it can also be divided according to the scene content type, such as city, suburb, mountainous area, river, etc.

[0029] In specific implementation, different area division rules can be configured for different scenarios based on the characteristics of the corresponding scenarios to improve the adaptability and flexibility of area division to the scenarios to be controlled, and to better adapt to a variety of different scenarios.

[0030] It should be noted that you can customize the regional division rules to divide the scene screen to be controlled into at least two scene areas. Subsequently, for each scene area, you can independently monitor the regional content information and realize independent sound effect control. This regionalized sound effect control strategy allows different scene areas to play different sound effects. For example, in the urban area, play the lively market background sound effects; in the suburban area, play the quiet ambient sound. Different from the global sound effect control, the regional division based on the scene screen to be controlled facilitates the dynamic control of local sound effects in large and complex scenes, and realizes the independent sound effect regulation of each scene area. Each scene area can independently control the sound effect level, which improves the level and dynamic expression of the sound effect.

[0031] Step 104: In response to the sound effect control event of the target scene area, determine the area content information of the target scene area, and determine the target sound effect parameters of the target scene area according to the area content information, wherein the target scene area is any one of the at least two scene areas.

[0032] Among them, the sound effect control event is an event that triggers the dynamic change of the sound effect control for the target scene area, such as the sound effect triggering event can be triggered regularly or based on the object information in the target scene area. The regional content information refers to the information related to the scene content type and object in the target scene area, such as the regional content information can include object information and / or scene content type.

[0033] Among them, the object may refer to a character in the scene to be controlled that may affect the sound effect control. For example, in a game scene, the object may be a player, an NPC (Non-Player Character), etc. The NPC character refers to a character that is not controlled by a real player in the game, but has a behavior pattern pre-set by a game program or script; in a video scene, the object may be a person included in the video screen; in an animation scene, the object may be a virtual animation character in the animation screen.

[0034] Sound effect parameters refer to various parameters used to control and adjust audio characteristics in audio processing to change the volume, sound quality, spatial sense, dynamic range, etc. of the audio, so as to make the audio more suitable for specific application scenarios. The target sound effect parameters are sound effect parameters determined based on the current regional content information of the target scene area. For example, the target sound effect parameters may include sound effect content and corresponding content control parameters. The sound effect content may include background music, ambient music and / or special effects music. The content control parameters may include volume, reverberation, number of instruments, etc.

[0035] In an optional implementation, a duration can be set for each interval to trigger a sound effect control event, automatically detect the regional content information of the target scene area, and determine the target sound effect parameters corresponding to the current regional content information. This allows changes in the regional content information of the target scene area to be detected in a timely manner, and dynamic changes in the regional content information to be responded to in a timely manner, thereby promptly determining the corresponding target sound effect parameters and achieving dynamic sound effect control.

[0036] In another optional implementation, object information in the target scene area can also be monitored, and the object information can be the number of objects, object behavior, etc. When the object information in the target scene area is monitored to meet the control conditions, it is determined to trigger a sound effect control event, detect the current area content information of the target scene area, and realize dynamic control of the sound effect. Among them, the control condition is a pre-configured condition for sound effect change, such as the control condition is an increase in the number of objects, a decrease in the number of objects, and / or objects entering and leaving the target scene area. At this time, the object information in the target scene area can be monitored. If it is monitored that the number of objects in the target scene area increases, the number of objects decreases, and / or objects enter and leave the target scene area, the sound effect control event is triggered. Under this implementation, the sound effect change can be dynamically triggered according to the object information to ensure smooth switching and timely response of the sound effect, and avoid the abruptness caused by frequent switching.

[0037] It should be noted that in response to the sound effect control event of the target scene area, the current regional content information of the target scene area can be determined, and the target sound effect parameters currently corresponding to the target scene area can be dynamically determined based on the current regional content information. By monitoring the real-time data in the target scene area, the corresponding sound effects can be dynamically adjusted to ensure that the sound effects are synchronized with the dynamically changing regional content information, making the sound effect control more real-time and flexible.

[0038] In an optional implementation of this embodiment, the region content information includes object information and / or scene content type; determining the region content information of the target scene region includes at least one of the following: Detecting the target scene area and determining object information of the target scene area; According to the scene identifier of the target scene area, the scene content type of the target scene area is determined.

[0039] In actual implementation, the area content information may include object information and / or scene content type, wherein the object information is used to indicate data related to objects in the target scene area, such as the number of objects, object behavior, object clothing color, etc. The scene content type is used to indicate the area type of the scene area, such as city, suburb, mountain, river, forest, etc.

[0040] It should be noted that after the scene image to be controlled is divided into at least two scene areas, the scene content type of each scene area can be determined, and a scene identifier can be marked for each scene area, and the scene identifier is used to indicate the scene content type of the corresponding scene area, such as city, suburb, mountain, river, forest, etc. In specific implementation, the scene identifier can be manually marked, or after the at least two scene areas are divided and determined, image recognition is performed on each scene area respectively to determine the scene content type, and then the corresponding scene identifier is automatically marked for each scene area.

[0041] In actual implementation, at least one dimension of the target scene area can be detected to obtain object information of at least one dimension of the target scene area; and the scene identification of the target scene area is read, and the scene content type of the target scene area is determined based on the scene identification. The object information and / or the scene content type are used as the area content information of the target scene area.

[0042] In the embodiments of the present specification, object information of the target scene area can be detected, and the scene content type can be determined as the area content information of the target scene area. Subsequently, corresponding sound effect control can be performed in combination with the object information and / or scene content type, thereby enriching the level of sound effect control, ensuring that the sound effect is synchronized with the scene content type and / or object behavior, and making the sound effect control more real-time and flexible.

[0043] In an optional implementation of this embodiment, the object information includes at least one of the number of objects, the behavior of the objects, and the color of the clothing of the objects; and detecting the target scene area to determine the object information of the target scene area includes at least one of the following: Perform edge detection on the target scene area, identify the object role outline, and determine the number of objects in the target scene area based on the identified object role outline; Performing motion detection on the target scene area, determining at least one motion track of at least one moving object in the target scene area, aggregating the motion tracks of the same moving object, and determining the number of objects in the target scene area; Performing motion detection on the target scene area, determining at least one motion track of at least one moving object in the target scene area, aggregating the motion tracks of the same moving object, obtaining motion parameters of each object, and using the motion parameters of each object as the object behavior; The color space of the target scene area is detected, the object role area is determined according to the set color threshold, and the object clothing color of each object in the target scene area is determined according to the object role area.

[0044] In actual implementation, the object information includes at least one of the number of objects, the behavior of the objects, and the color of the clothing of the objects.

[0045] In the first implementation, the object information may include the number of objects, edge detection is performed on the target scene area, the outline of the object role is identified, and the number of objects in the target scene area is determined based on the identified outline of the object role. Specifically, the edge in the image of the target scene area can be extracted using a set edge detection algorithm, and then the shape of the object role can be identified through contour analysis. Based on the set shape characteristics of the object role (such as area, aspect ratio, etc.), noise is filtered, and shapes (noise) that are too small or too large are excluded. The outline that meets the characteristics of the object role is retained, and then the number of objects in the target scene area is dynamically determined through contour analysis and statistics.

[0046] Among them, the set edge detection algorithm can be any edge detection algorithm, such as Sobel operator, Prewitt operator, Canny edge detection, Laplacian operator, Roberts cross gradient operator, etc. Sobel, Prewitt and Roberts operators are suitable for scenes that need to detect edges quickly. Although the Canny edge detection algorithm has high computational complexity, it can provide more accurate edge detection results and is suitable for scenes with high precision requirements. The Laplacian operator is sensitive to noise and is suitable for use on images that have been fully denoised in the preprocessing stage. In actual applications, the corresponding edge detection algorithm can be selected according to needs.

[0047] After using the set edge detection algorithm to extract the edges in the target scene area, you can use the cv2.findContours() function in the OpenCV library to perform contour detection and filter out contour areas that do not meet the conditions based on area and aspect ratio. Count the contours that are finally determined to obtain the number of objects in the target scene area.

[0048] In the second implementation, the object information may include the number of objects, and at least one motion track of at least one moving object in the target scene area may be determined through motion detection, and the various motion tracks of the same moving object may be aggregated to determine the number of objects in the target scene area. Specifically, based on a set optical flow detection method, a set number of continuous frames of the target scene area before the current time may be tracked, and the pixel motion in the continuous frames may be tracked, including the motion direction and speed. The trajectories of these moving objects may be calculated through the optical flow method, and whether they are object roles may be determined based on parameters such as the size and motion speed of the moving objects. For example, objects usually have obvious moving directions and speeds in the scene picture, while the background is relatively still. At least one motion track of at least one moving object may be obtained through tracking, and the motion paths of different moving objects may be determined through a trajectory clustering algorithm. The various motion tracks of the same moving object may be aggregated to determine the number of objects in the target scene area.

[0049] The optical flow detection method set can be any optical flow detection method, such as the Horn-Schunck method, the Lucas-Kanade method, and the Farnebäck method. The Horn-Schunck method is a global method that estimates the optical flow field by minimizing an energy function. This method assumes that the optical flow is smooth over the entire image. The energy function includes a data term (brightness constant assumption) and a regularization term (smoothness assumption). The optical flow field is solved by iterative optimization. The Lucas-Kanade method is a local method suitable for small motion situations. It assumes that the optical flow is constant in a small window. The optical flow of each pixel is estimated by solving the optical flow equations. Specifically, it uses the brightness constant assumption and Taylor expansion to linearize the optical flow equations, and then solves them by the least squares method. The Farnebäck method is a multi-scale method that estimates the optical flow at multiple scales and gradually refines the results. It combines the advantages of the Horn-Schunck method and the Lucas-Kanade method and can provide accurate optical flow estimation within a larger range of motion. In practical applications, the corresponding optical flow detection method can be selected according to needs.

[0050] The Lucas-Kanade method is a widely used optical flow algorithm suitable for real-time tracking of multiple moving objects. The movement behavior of the object is determined by calculating the motion trajectory in the optical flow field. The Lucas-Kanade method is taken as an example in the embodiments of this specification. OpenCV provides an implementation of the Lucas-Kanade optical flow method. The cv2.calcOpticalFlowPyrLK() function in OpenCV can be used to calculate the optical flow field for motion tracking.

[0051] Based on the set optical flow detection method, at least one motion trajectory of at least one moving object can be tracked and identified. Then, based on the clustering algorithm of the motion trajectory, the various motion trajectories can be grouped, and the various motion trajectories of the same moving object can be aggregated together, such as K-Means clustering (distance-based clustering algorithm), DBSCAN (Density-Based Spatial Clustering of Applications with Noise, density-based clustering algorithm), HDBSCAN (Hierarchical DBSCAN, automatically determines the optimal ε value and number of clusters through hierarchical clustering method), OPTICS (Ordering Points To Identify the Clustering Structure, reveals the clustering structure of data by generating a reachable distance graph), Gaussian Mixture Models (GMM, clustering algorithm based on probability model), etc. The corresponding clustering algorithm can be selected based on actual needs.

[0052] It should be noted that the method of determining the number of objects based on edge detection and contour recognition is suitable for scene areas with a large number of static objects, while the method of determining the number of objects based on motion detection is suitable for scene areas with dynamic objects, especially in complex scenes, and can effectively distinguish static backgrounds from dynamic objects. In practical applications, the number of objects can be determined based on the actual situation of the target scene area, using a method based on edge detection and contour recognition or a method based on motion detection. Of course, the number of objects can also be determined by combining the above two implementation methods at the same time. The two determination methods can be verified with each other to improve the accuracy of the number of objects.

[0053] In a third implementation, the object information may include object behavior, and motion detection may be performed on the target scene area to determine at least one motion trajectory of at least one moving object in the target scene area, aggregate the motion trajectories of the same moving object, obtain the motion parameters of each object, and use the motion parameters of each object as the object behavior.

[0054] Among them, motion detection is performed on the target scene area, at least one motion trajectory of at least one moving object in the target scene area is determined, and the motion trajectories of the same moving object are aggregated. The specific implementation process is the same as that in the second implementation method mentioned above, and the embodiments of this specification will not be repeated here.

[0055] The optical flow detection method can track the movement direction and speed of a moving object, and thus aggregating the various motion trajectories of the same moving object can obtain the motion trajectory of each object. The motion trajectory can indicate the motion parameters of the corresponding object, and the motion parameters are used to indicate the movement of the object in the target scene area in a set number of consecutive frames for optical flow tracking, such as the number of motion trajectories, motion direction and / or motion speed, etc. The determined motion parameters can be used as object behavior, which is convenient for subsequent different sound effect control based on different object behaviors.

[0056] In a fourth implementation, the object information may include the object clothing color, the color space of the target scene area is detected, the object role area is determined according to the set color threshold, and the object clothing color of each object in the target scene area is determined according to the object role area. Specifically, the image of the target scene area can be converted into the HSV color space, which is easier to process colors with different brightness and saturation. The main color area is filtered out by setting the color threshold, the object role area is obtained, and the object clothing color of each object is determined, so that different sound effect controls can be achieved based on different object clothing colors in the future.

[0057] Among them, the set color threshold is the color that the pre-set character clothing may correspond to. For example, in a multiplayer game, assuming that the object characters in the game wear uniform clothing colors, such as red or blue, these pixel areas can be retained by setting the corresponding HSV range, retaining these specific color pixel areas, identifying the object character area, and determining the specific object clothing color based on the specific HSV range, effectively detecting the distribution of the object characters in the scene and identifying the object clothing color. In actual implementation, after converting the image of the target scene area to the HSV color space, the cv2.inRange() function in the OpenCV library can be used to filter the color.

[0058] It should be noted that HSV color space conversion and shape contour analysis can be used to accurately identify and track the object roles in the scene to achieve efficient sound effect control; optical flow detection technology can be used to monitor the dynamic motion trajectory in the target scene area, automatically identify the behavior of multiple objects and dynamically adjust the sound effect level according to the number of objects; In the embodiments of this specification, HSV color space conversion and shape contour analysis can be used to accurately identify and track the object roles in the scene to achieve efficient sound effect control; optical flow detection technology is used to monitor the dynamic motion trajectory in the target scene area, automatically identify the behavior of multiple objects, and dynamically adjust the sound effect level according to the number of objects. In this way, object shape analysis, optical flow motion detection, color detection, etc. are used to realize the identification of the number of objects, object behavior, and object clothing color, and real-time monitoring of multi-dimensional object information such as the number of objects, object behavior, and object clothing color, so that subsequent multi-dimensional monitoring results based on the pre-configured sound effect control rules can be combined to dynamically control the sound effects of the target scene area, making the sound effect control more real-time and flexible.

[0059] In an optional implementation of this embodiment, the region content information includes object information and / or scene content type; determining the target sound effect parameters of the target scene region according to the region content information includes: According to the object information and / or scene content type, in combination with the pre-configured sound effect control rules, the target sound effect parameters of the target scene area are determined, wherein the sound effect control rules are configured with sound effect parameters corresponding to different object information and / or scene content types.

[0060] Among them, the regional content information may include object information and / or scene content type, and the sound effect control rule refers to the pre-configured correspondence rule between object information and / or scene content type and sound effect parameters, that is, the sound effect control rule can indicate what kind of sound effect parameters correspond to each object information and / or scene content type.

[0061] It should be noted that detailed sound effect control rules can be configured based on actual control needs to define what kind of sound effect parameters correspond to each object information and / or scene content type. The definition of sound effect control rules can be implemented specifically through a logic rule engine. After determining the object information and / or scene content type of the target scene area, you can query the pre-configured sound effect control rules to determine the target sound effect parameters of the target scene area, so that the sound effect control of the target scene area can be implemented based on the target sound effect parameters. For example, the urban area corresponds to the lively market background sound effects, and the suburban area corresponds to the quiet ambient sound.

[0062] In the embodiments of this specification, detailed sound effect control rules can be set to adjust the sound effect control parameters such as the complexity, volume and effect of the sound effects in the target scene area according to the object information and / or the scene content type. This rule-based sound effect control mechanism is different from the traditional event trigger mechanism, and is more flexible and responsive. It can be applied to complex multiplayer game scenes such as multiplayer online games, open world games, and story-driven games, such as combat and exploration. It can also be applied to other complex scenes such as videos and multiplayer animations, providing diverse sound effect performances and enhancing the expressiveness of sound effects in various scenes.

[0063] In an optional implementation of this embodiment, the target sound effect parameters of the target scene area are determined according to the object information and / or the scene content type in combination with the pre-configured sound effect control rules, including: In the case where the area content information includes object information, and the object information includes the number of objects, determining the target number range to which the number of objects belongs in combination with a pre-configured sound effect control rule; A target sound effect parameter corresponding to a target quantity range is determined from the sound effect control rule, wherein different quantity ranges in the sound effect control rule correspond to different sound effect parameters.

[0064] In one implementation, the regional content information includes object information, and the object information includes the number of objects. The sound effect control rule can be configured with sound effect parameters corresponding to multiple different quantity ranges, and different quantity ranges correspond to different sound effect parameters. At this time, the target quantity range to which the object quantity belongs in the sound effect control rule can be determined, and then the target sound effect parameters corresponding to the target quantity range can be queried from the sound effect control rule.

[0065] It should be noted that the sound effect control rules can be pre-configured with sound effect parameters corresponding to different quantity ranges. The larger the quantity range of objects, the richer the sound effect levels corresponding to the sound effect parameters can be configured, that is, the higher the complexity of the sound effect. For example, the larger the quantity range, the more types of sound effect content it contains, such as increasing ambient music, special effects music, etc. as the number of objects increases; and / or, the larger the quantity range, the more audio tracks corresponding to the sound effect content, such as adding more audio tracks, such as percussion instruments, vocals, etc., as the number of objects increases, to enhance the overall sense of hierarchy of the sound effect.

[0066] For example, the pre-configured sound effect control rule is that when the number of objects is greater than 10, the system plays rich background music and ambient music. At this time, the corresponding sound effect parameters include 2 types of sound effect contents, including background sound effects and ambient music, and the number of audio tracks of each sound effect content is X1; when the number of objects is between 5 and 10, medium-complexity sound effects are played. At this time, the corresponding sound effect parameters include 2 types of sound effect contents, including background music and ambient music, and the number of audio tracks of each sound effect content is X2, and X2 is less than X1; when the number of objects is less than 5, simplified low-volume background sound effects are played. At this time, the corresponding sound effect parameters include 1 type of sound effect content, including background music, and the number of audio tracks of background music is X3, and X3 is less than or equal to X2.

[0067] Assuming that the number of objects in the target scene area is 8, the target number range is determined to be between 5 and 10. At this time, the target sound effect parameters can be determined as 2 types of sound effect content, including background music and ambient music, and the number of audio tracks of each sound effect content is X2.

[0068] In the embodiments of the present specification, different numbers of objects may correspond to different sound effect parameters. By real-time monitoring of the number of objects in the target scene area, the corresponding sound effect parameters may be dynamically adjusted. For example, the more objects there are, the more complex the sound effect may be. By real-time monitoring of the number of objects and dynamically adjusting the sound effect, the sound effect is more closely aligned with the scene, greatly enhancing the immersive experience of the object.

[0069] In an optional implementation of this embodiment, the target sound effect parameters of the target scene area are determined according to the object information and / or the scene content type in combination with the pre-configured sound effect control rules, including: When the regional content information includes object information, and the object information includes the object's clothing color, the target sound effect parameters corresponding to the object's clothing color are determined in combination with the pre-configured sound effect control rules, wherein different object clothing colors in the sound effect control rules correspond to different sound effect parameters.

[0070] In another implementation, the area content information includes object information, and the object information includes the object's clothing color. The sound effect control rule can be configured with sound effect parameters corresponding to various clothing colors, and different object clothing colors correspond to different sound effect parameters.

[0071] In actual implementation, if the object clothing colors of multiple objects in the target scene area are different, the object clothing color of the target scene area can be determined based on pre-defined screening rules, such as selecting the color with the most colors as the object clothing color of the target scene area, or selecting the color with the largest density as the object clothing color of the target scene area, or selecting the color closest to the center of the target scene area as the object clothing color of the target scene area.

[0072] For example, suppose the pre-configured sound effect control rule is that red clothing corresponds to intense and cheerful sound effects, blue clothing corresponds to steady and slow sound effects, etc. Suppose the color of the clothing of the object in the determined target scene area is red, then the corresponding target sound effect parameter can be determined to be an intense and cheerful sound effect.

[0073] In the embodiments of the present specification, different colors of the object's clothing may correspond to different sound effect parameters. By real-time monitoring of the object's clothing color in the target scene area, the corresponding sound effect parameters may be dynamically adjusted. For example, the brighter the object's clothing color, the more complex and cheerful the sound effect may be. By real-time monitoring of the object's clothing color and dynamically adjusting the sound effect, the sound effect and the scene are more closely matched, thereby greatly enhancing the object's immersive experience.

[0074] In an optional implementation of this embodiment, the target sound effect parameters of the target scene area are determined according to the object information and / or the scene content type in combination with the pre-configured sound effect control rules, including: In a case where the region content information includes object information, and the object information includes object behavior, determining motion parameters indicated by the object behavior, wherein the motion parameters include the number of motion trajectories, motion direction and / or motion speed; The corresponding target sound effect parameters are determined according to the motion parameters in combination with the pre-configured sound effect control rules, wherein different motion parameters in the sound effect control rules correspond to different sound effect parameters.

[0075] In another implementation, the area content information includes object information, and the object information includes object behavior. The object behavior can indicate the motion parameters of the corresponding object, such as the number of motion trajectories, motion direction and / or motion speed, etc. The sound effect control rules can be pre-configured with sound effect parameters corresponding to different motion parameters, and different motion parameters correspond to different sound effect parameters. At this time, the number of motion trajectories, motion direction and / or motion speed indicated by the object behavior of the target scene area can be determined, and the corresponding target sound effect parameters can be queried in the sound effect control rules.

[0076] It should be noted that the sound effect control rules can be pre-configured with sound effect parameters corresponding to different motion parameters. For example, the more motion trajectories there are, the more objects are moving in the target scene area. The sound effect parameters can be configured to have richer sound effect levels and louder volume. For example, as the number of motion trajectories increases, the types of sound effect content, the levels of background music, the number of tracks, etc. are increased, such as adding percussion or reverberation effects; and / or, different sound effect parameters are configured to correspond to different motion directions. For example, the volume for movement to the left is Y1, and the volume for movement to the right is Y2, where Y1 is different from Y2, or different sound effect content, tracks, etc. can be configured for different motion directions; and / or, different sound effect parameters are configured to correspond to different motion speeds. For example, the greater the motion speed, the faster the object moves in the target scene area. The sound effect parameters can be configured to have richer sound effect levels and louder volume. For example, as the motion speed increases, the types of sound effect content, the levels of background music, the number of tracks, etc. are increased, such as adding percussion or reverberation effects.

[0077] In the embodiments of the present specification, different object behaviors may correspond to different sound effect parameters. By real-time monitoring of the number of motion trajectories, motion directions and / or motion speeds in the target scene area, the corresponding sound effect parameters may be dynamically adjusted. By real-time monitoring of the number of motion trajectories, motion directions and / or motion speeds, the sound effects may be dynamically adjusted, so that the sound effects are more closely aligned with the scene, greatly enhancing the immersive experience of the object.

[0078] It should be noted that the above-mentioned multiple sound effect control methods can be used in combination based on actual business needs. For example, the sound effect control can be combined with the scene content type and the number of objects. In urban areas, when a large number of objects are detected, a lively market background sound effect is played; while in suburban areas, when objects are sparse, a quiet ambient sound is played. In other words, when dense objects are detected in urban areas, the complexity of the background music can be increased, such as adding more sound effect layers and instrument tracks; while in areas with fewer objects, simpler background sounds are used.

[0079] Each scene area can dynamically adjust the sound effect parameters according to the scene content type, number of objects, object behavior, and object clothing color to enhance the layering and dynamic expression of the sound effects.

[0080] For example, taking the game scene as an example, in an open world game, the sound effects can change dynamically as objects gather and evacuate to enhance the sense of immersion; in a multiplayer online game, the sound effects of the battlefield area can be adjusted based on the distribution of objects to make the sound effects more in line with the scene changes; in the immersive experience, a more realistic environmental atmosphere can be created through dynamic sound effect changes.

[0081] Step 106: Control the sound effect of the target scene area according to the target sound effect parameters of the target scene area.

[0082] It should be noted that the target sound effect parameters may include sound effect content and corresponding content control parameters. The sound effect content may include background music, ambient music and / or special effects music, that is, the complexity of the sound effect content can be adjusted. The content control parameters may include volume, reverberation, number of instruments, etc.

[0083] In actual implementation, after the target sound effect parameters of the target scene area are determined, the sound effect of the target scene area can be controlled according to the target sound effect parameters to achieve the corresponding sound effect.

[0084] In an optional implementation of this embodiment, controlling the sound effect of the target scene area according to the target sound effect parameter of the target scene area includes: Determine historical sound effect parameters of the target scene area; Determining transition parameters from historical sound effect parameters to target sound effect parameters; According to the transition parameters, the sound effects in the target scene area are controlled to transition from the historical sound effect parameters to the target sound effect parameters.

[0085] It should be noted that switching the sound effect parameters of the target scene area from historical sound effect parameters to determined target sound effect parameters may produce an abrupt feeling and affect the user experience. Therefore, the historical sound effect parameters of the target scene area, that is, the sound effect parameters before the switch, can be determined, and then based on the time gradient technology, the transition parameters from the historical sound effect parameters to the target sound effect parameters can be determined. Based on the transition parameters, the sound effects of the target scene area are controlled to gradually transition from the historical sound effect parameters to the target sound effect parameters. For example, when the number of objects is detected to increase from 5 to 10, the volume of the background sound, the reverberation effect and the number of musical instruments are gradually enhanced to avoid the discomfort caused by the sudden sound effect switch.

[0086] Among them, temporal blending is a commonly used technique in computer graphics and animation, which is used to smoothly transition or mix states at different time points. This technique can be applied to a variety of scenarios, such as smooth transition between animation frames, color gradient, path following, etc. The main purpose of temporal blending is to reduce mutations and make the animation or effect look more natural and smooth. The core idea of ​​temporal blending is to insert an intermediate state between two or more known states so that there is a smooth transition from one state to another. This can be achieved through methods such as linear interpolation (LEP) and spline interpolation.

[0087] Linear interpolation is a simple numerical interpolation method used to estimate the value of an unknown point between known data points. Linear interpolation assumes that the function between two points is linear, so the value of the middle point can be calculated by the coordinates of the two points, that is, the sound effect parameters are in numerical form, such as volume, reverberation, number of instruments, etc. Based on the values ​​of historical sound effect parameters and target sound effect parameters, multiple intermediate values ​​can be calculated as transition parameters. For example, if the volume of the historical sound effect parameters is 50 and the volume of the target sound effect parameters is 80, multiple intermediate values ​​of 55, 60, 65, 70, and 75 are determined by linear interpolation. At this time, 60, 65, 70, and 75 can be used as transition volumes to achieve a smooth transition from 50 to 80 in the target scene area according to 55, 60, 65, 70, and 75.

[0088] In the embodiments of this specification, through the time gradient technology, the sound effects can be smoothly adjusted when the sound effect parameters of the target scene area change, avoiding abrupt switching, ensuring that the sound effects of the target scene area transition smoothly with the changes in the regional content information, providing smoother and more natural sound effect changes, and improving the user's immersive experience.

[0089] In an optional implementation of this embodiment, the target sound effect parameters include sound effect content and corresponding content control parameters, and the sound effect content includes background music, ambient music and / or special effect music; and controlling the sound effect of the target scene area according to the target sound effect parameters of the target scene area includes: Determine the audio tracks corresponding to the background music, ambient music and / or special effect music respectively; Based on the corresponding sound effect control parameters, the audio tracks corresponding to the background music, ambient music and / or special effect music are controlled.

[0090] It should be noted that the target sound effect parameters may include sound effect content and corresponding content control parameters. The sound effect content includes background music, ambient music and / or special effect music. The more types of sound effect content included, the higher the complexity of the sound effect. The content control parameters are parameters of the corresponding sound effect content, such as volume, reverberation, number of instruments, etc.

[0091] In actual implementation, after determining the target sound effect parameters of the target scene area, the target sound effect parameters include background music, ambient music and / or special effects music. At this time, the audio tracks corresponding to the background music, ambient music and / or special effects music can be determined respectively, and the audio tracks corresponding to the background music, ambient music and / or special effects music can be controlled respectively. According to the corresponding sound effect control parameters, the audio of the corresponding effect is played to achieve the corresponding sound effect.

[0092] In the embodiments of this specification, a multi-layer sound effect track strategy is adopted to process the sound tracks corresponding to background music, ambient music, and special effect music in layers, and gradually add or reduce the sound effect levels; and, through the sub-track sound effect controller, the volume and effects of different sound tracks are adjusted in real time according to the sound effect control parameters corresponding to the background music, ambient music, and / or special effect music, and the content and effects of the sound tracks are gradually increased or reduced. In this way, through the mixing of multi-track sound effects, a rich sound effect level can be provided according to the regional content information in the target scene area, achieving the richness of the sound effect level, making the sound effects more realistic and diverse in complex scenes, and further improving the expressiveness.

[0093] In an optional implementation of this embodiment, the target sound effect parameter also includes a track ratio; respectively determining the tracks corresponding to the background music, the ambient music and / or the special effect music includes: Based on the target sound effect parameters, determine the proportion of audio tracks corresponding to background music, ambient music and / or special effect music; According to the proportion of the audio tracks, the audio tracks corresponding to the background music, the ambient music and / or the special effect music are determined respectively.

[0094] In actual implementation, the target sound effect parameters also include the track ratio. At this time, the track ratio corresponding to the background music, ambient music and / or special effects music can be determined based on the target sound effect parameters, and the tracks corresponding to the background music, ambient music and / or special effects music can be determined respectively. For example, the number of tracks corresponding to background music: ambient music: special effects music is 1:5:3. At this time, it can be determined that the background music corresponds to 1 track, the ambient music corresponds to 5 tracks, and the special effects music corresponds to 3 tracks.

[0095] For example, when the number of objects in the target scene area increases, the percussion and vocal tracks in the background music may be increased, that is, the proportion of the number of background music tracks in the target sound effect parameters increases.

[0096] It should be noted that when the regional content information of the target scene area changes, the determined target sound effect parameters will also change accordingly. The target sound effect parameters can be the audio track ratio. Therefore, the ratio of different audio tracks can be dynamically adjusted according to the regional content information of the target scene area to enhance the overall sense of hierarchy of the sound effect.

[0097] The embodiment of the present specification provides a sound effect control method, which realizes dividing the scene picture to be controlled into at least two scene areas, and for any target scene area in the at least two scene areas, in response to the sound effect control event of the target scene area, the regional content information of the target scene area is determined from multiple dimensions, and the target sound effect parameters of the target scene area are determined based on the currently determined regional content information, so as to realize the sound effect control of the target scene area. In this way, by dividing the scene picture to be controlled into regions, the sound effects of different scene areas can be independently regulated, and the changes in regional content information can be flexibly responded to, allowing different scene areas to play different sound effects, enriching the sound effect content, and improving the layering and dynamic expression of the sound effect; and, for independent scene areas, by identifying the current multi-dimensional regional content information to determine the corresponding target sound effect parameters, dynamic sound effect control can be realized based on the content information of multiple dimensions of the scene dynamic changes, ensuring that the sound effect is synchronized with the content information change of the scene area, avoiding the sound effect being out of touch with the actual scene in complex scenes, improving the control effect of the sound effect, and thus improving the immersion and experience of the object.

[0098] The following combination Figure 2 , taking the application of the sound effect control method provided in this specification in a game scenario as an example, the sound effect control method is further described. Figure 2 FIG. 1 shows a processing flow chart of a sound effect control method provided by an embodiment of the present specification, such as Figure 2 As shown, the game scene is divided into multiple scene areas.

[0099] Any one of the multiple scene areas is used as the target scene area for color detection. Specifically, the HSV color space and the color threshold can be used to extract the object character area and determine the object clothing color of each object. The object can be a player and / or NPC character.

[0100] In addition, shape detection can be performed on the target scene area. Specifically, edge detection and contour analysis can be used to filter noise based on the shape characteristics of the set object role (such as area, aspect ratio, etc.), retain contours that meet the characteristics of the object role, and dynamically determine the number of objects in the scene.

[0101] In addition, motion detection may be performed for the target scene area. Specifically, an optical flow detection method and a trajectory clustering algorithm may be used to determine the number of objects in the target scene area; or motion parameters may be determined based on the optical flow detection method and the trajectory clustering algorithm.

[0102] Based on the object clothing color, object number, motion parameters, and the regional content type of the target scene area, the corresponding target sound effect parameters are determined to achieve dynamic control of the sound effects in the target scene area.

[0103] Corresponding to the above method embodiment, this specification also provides a sound effect control device embodiment, Figure 3 FIG. 1 is a schematic diagram showing the structure of a sound effect control device provided by an embodiment of the present specification. Figure 3 As shown, the device comprises: A division module 302 is configured to divide the scene picture to be controlled into at least two scene areas; The determination module 304 is configured to determine the region content information of the target scene region in response to the sound effect control event of the target scene region, and determine the target sound effect parameter of the target scene region according to the region content information, wherein the target scene region is any one of the at least two scene regions; The control module 306 is configured to control the sound effect of the target scene area according to the target sound effect parameters of the target scene area.

[0104] Optionally, the area content information includes object information and / or scene content type; the determination module 304 is further configured to perform at least one of the following: Detecting the target scene area and determining object information of the target scene area; According to the scene identifier of the target scene area, the scene content type of the target scene area is determined.

[0105] Optionally, the object information includes at least one of the number of objects, the behavior of the objects, and the color of the clothing of the objects; the determination module 304 is further configured to perform at least one of the following: Perform edge detection on the target scene area, identify the object role outline, and determine the number of objects in the target scene area based on the identified object role outline; Performing motion detection on the target scene area, determining at least one motion track of at least one moving object in the target scene area, aggregating the motion tracks of the same moving object, and determining the number of objects in the target scene area; Performing motion detection on the target scene area, determining at least one motion track of at least one moving object in the target scene area, aggregating the motion tracks of the same moving object, obtaining motion parameters of each object, and using the motion parameters of each object as the object behavior; The color space of the target scene area is detected, the object role area is determined according to the set color threshold, and the object clothing color of each object in the target scene area is determined according to the object role area.

[0106] Optionally, the region content information includes object information and / or scene content type; the determination module 304 is further configured to: According to the object information and / or scene content type, in combination with the pre-configured sound effect control rules, the target sound effect parameters of the target scene area are determined, wherein the sound effect control rules are configured with sound effect parameters corresponding to different object information and / or scene content types.

[0107] Optionally, the determination module 304 is further configured to: In the case where the area content information includes object information, and the object information includes the number of objects, determining the target number range to which the number of objects belongs in combination with a pre-configured sound effect control rule; A target sound effect parameter corresponding to a target quantity range is determined from the sound effect control rule, wherein different quantity ranges in the sound effect control rule correspond to different sound effect parameters.

[0108] Optionally, the determination module 304 is further configured to: When the regional content information includes object information, and the object information includes the object's clothing color, the target sound effect parameters corresponding to the object's clothing color are determined in combination with the pre-configured sound effect control rules, wherein different object clothing colors in the sound effect control rules correspond to different sound effect parameters.

[0109] Optionally, the determination module 304 is further configured to: In a case where the region content information includes object information, and the object information includes object behavior, determining motion parameters indicated by the object behavior, wherein the motion parameters include the number of motion trajectories, motion direction and / or motion speed; The corresponding target sound effect parameters are determined according to the motion parameters in combination with the pre-configured sound effect control rules, wherein different motion parameters in the sound effect control rules correspond to different sound effect parameters.

[0110] Optionally, the control module 306 is further configured to: Determine historical sound effect parameters of the target scene area; Determining transition parameters from historical sound effect parameters to target sound effect parameters; According to the transition parameters, the sound effects in the target scene area are controlled to transition from the historical sound effect parameters to the target sound effect parameters.

[0111] Optionally, the target sound effect parameters include sound effect content and corresponding content control parameters, and the sound effect content includes background music, ambient music and / or special effect music; the control module 306 is further configured to: Determine the audio tracks corresponding to the background music, ambient music and / or special effect music respectively; Based on the corresponding sound effect control parameters, the audio tracks corresponding to the background music, ambient music and / or special effect music are controlled.

[0112] Optionally, the target sound effect parameter also includes a track ratio; the control module 306 is further configured to: Based on the target sound effect parameters, determine the proportion of audio tracks corresponding to background music, ambient music and / or special effect music; According to the proportion of the audio tracks, the audio tracks corresponding to the background music, the ambient music and / or the special effect music are determined respectively.

[0113] The embodiment of the present specification provides a sound effect control device, including a division module, a determination module and a control module. Through the interaction and cooperation of each module in the sound effect control device, the scene screen to be controlled is divided into at least two scene areas. For any target scene area in the at least two scene areas, in response to the sound effect control event of the target scene area, the regional content information of the target scene area is determined, and the target sound effect parameters of the target scene area are determined based on the currently determined regional content information, so as to realize the sound effect control of the target scene area. In this way, the regional content information can be detected independently by region, so as to realize the sound effect control of the scene area, and the sound effects of different scene areas are independently controlled, allowing different scene areas to play different sound effects, enriching the sound effect content, and improving the layering and dynamic expression of the sound effect; and for independent scene areas, by identifying the current regional content information to determine the corresponding target sound effect parameters, dynamic sound effect control can be realized based on the dynamically changing regional content of the scene, ensuring that the sound effect changes synchronously with the content information of the scene area, avoiding the sound effect being out of touch with the actual scene in complex scenes, improving the control effect of the sound effect, and thus improving the immersion and experience of the object.

[0114] The above is a schematic scheme of a sound effect control device of this embodiment. It should be noted that the technical scheme of the sound effect control device and the technical scheme of the above sound effect control method belong to the same concept, and the details not described in detail in the technical scheme of the sound effect control device can be referred to the description of the technical scheme of the above sound effect control method.

[0115] Figure 4 The block diagram of a computing device according to an embodiment of the present specification is shown. The components of the computing device 400 include but are not limited to a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and the database 450 is used to store data.

[0116] The computing device 400 also includes an access device 440 that enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0117] In one embodiment of the present specification, the above components of the computing device 400 and Figure 4 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 4 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0118] The computing device 400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 400 may also be a mobile or stationary server.

[0119] The processor 420 is used to execute the following computer executable instructions, which implement the steps of the above-mentioned sound effect control method when executed by the processor.

[0120] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned sound effect control method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned sound effect control method.

[0121] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which can implement the steps of the above-mentioned sound effect control method when executed by a processor.

[0122] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned sound effect control method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned sound effect control method.

[0123] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned sound effect control method.

[0124] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above-mentioned sound effect control method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be referred to the description of the technical scheme of the above-mentioned sound effect control method.

[0125] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0126] Computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. Computer readable media may include: any entity or device capable of carrying computer program codes, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memories, read-only memories (ROM), random access memories (RAM), electric carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of computer readable media may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0127] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0128] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0129] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A sound effect control method, characterized in that: include: Divide the scene picture to be controlled into at least two scene areas; In response to a sound effect control event of a target scene area, determining area content information of the target scene area, and determining a target sound effect parameter of the target scene area according to the area content information, wherein the target scene area is any one of the at least two scene areas; The sound effect of the target scene area is controlled according to the target sound effect parameters of the target scene area.

2. The sound effect control method according to claim 1, characterized in that: The area content information includes object information and / or scene content type; the determining of the area content information of the target scene area includes at least one of the following: Detecting the target scene area to determine object information of the target scene area; The scene content type of the target scene area is determined according to the scene identifier of the target scene area.

3. The sound effect control method according to claim 2, characterized in that: The object information includes at least one of the number of objects, the behavior of the objects, and the color of the clothing of the objects; the detecting the target scene area to determine the object information of the target scene area includes at least one of the following: Performing edge detection on the target scene area to identify the contours of the object roles, and determining the number of objects in the target scene area according to the identified contours of the object roles; Performing motion detection on the target scene area to determine at least one motion track of at least one moving object in the target scene area, aggregating the motion tracks of the same moving object, and determining the number of objects in the target scene area; Performing motion detection on the target scene area, determining at least one motion trajectory of at least one moving object in the target scene area, aggregating the motion trajectories of the same moving object, obtaining motion parameters of each object, and using the motion parameters of each object as the object behavior; The color space of the target scene area is detected, an object role area is determined according to a set color threshold, and the object clothing color of each object in the target scene area is determined according to the object role area.

4. The sound effect control method according to claim 1, characterized in that: The area content information includes object information and / or scene content type; and determining the target sound effect parameters of the target scene area according to the area content information includes: According to the object information and / or scene content type, combined with pre-configured sound effect control rules, target sound effect parameters of the target scene area are determined, wherein the sound effect control rules are configured with sound effect parameters corresponding to different object information and / or scene content types.

5. The sound effect control method according to claim 4, characterized in that: Determining the target sound effect parameters of the target scene area according to the object information and / or scene content type in combination with a pre-configured sound effect control rule includes: In a case where the area content information includes object information, and the object information includes the number of objects, determining a target number range to which the number of objects belongs in combination with a pre-configured sound effect control rule; The target sound effect parameters corresponding to the target quantity range are determined from the sound effect control rule, wherein different quantity ranges in the sound effect control rule correspond to different sound effect parameters.

6. The sound effect control method according to claim 4, characterized in that: Determining the target sound effect parameters of the target scene area according to the object information and / or scene content type in combination with a pre-configured sound effect control rule includes: In a case where the area content information includes object information, and the object information includes the object's clothing color, the target sound effect parameters corresponding to the object's clothing color are determined in combination with a preconfigured sound effect control rule, wherein different object clothing colors in the sound effect control rule correspond to different sound effect parameters.

7. The sound effect control method according to claim 4, characterized in that: Determining the target sound effect parameters of the target scene area according to the object information and / or scene content type in combination with a pre-configured sound effect control rule includes: In a case where the region content information includes object information, and the object information includes object behavior, determining a motion parameter indicated by the object behavior, wherein the motion parameter includes the number of motion trajectories, a motion direction and / or a motion speed; The corresponding target sound effect parameters are determined according to the motion parameters in combination with the pre-configured sound effect control rules, wherein different motion parameters in the sound effect control rules correspond to different sound effect parameters.

8. The sound effect control method according to any one of claims 1 to 7, characterized in that: The controlling the sound effect of the target scene area according to the target sound effect parameter of the target scene area comprises: Determining historical sound effect parameters of the target scene area; Determining transition parameters from the historical sound effect parameters to the target sound effect parameters; According to the transition parameter, the sound effect of the target scene area is controlled to transition from the historical sound effect parameter to the target sound effect parameter.

9. The sound effect control method according to any one of claims 1 to 7, characterized in that: The target sound effect parameters include sound effect content and corresponding content control parameters, wherein the sound effect content includes background music, ambient music and / or special effect music; and controlling the sound effect of the target scene area according to the target sound effect parameters of the target scene area includes: Determine the audio tracks corresponding to the background music, the ambient music and / or the special effect music respectively; Based on the corresponding sound effect control parameters, the audio tracks corresponding to the background music, the ambient music and / or the special effect music are controlled.

10. The sound effect control method according to claim 9, characterized in that: The target sound effect parameter also includes a track ratio; and the step of respectively determining the tracks corresponding to the background music, the ambient music and / or the special effect music includes: Based on the target sound effect parameter, determining the proportion of audio tracks corresponding to the background music, the ambient music and / or the special effect music; According to the audio track ratio, the audio tracks corresponding to the background music, the ambient music and / or the special effect music are respectively determined.

11. A sound effect control device, characterized in that: include: A division module, configured to divide the scene picture to be controlled into at least two scene areas; a determination module configured to determine, in response to a sound effect control event of a target scene area, area content information of the target scene area, and determine a target sound effect parameter of the target scene area according to the area content information, wherein the target scene area is any one of the at least two scene areas; The control module is configured to control the sound effect of the target scene area according to the target sound effect parameters of the target scene area.

12. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the sound effect control method described in any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, implement the steps of the sound effect control method described in any one of claims 1 to 10.

14. A computer program product, characterized in that It includes a computer program / instruction, which, when executed by a processor, implements the steps of the sound effect control method described in any one of claims 1 to 10.

Images