Audio processing method and device, electronic equipment and medium
The image collector determines the object distance and orientation in the audio device, and adjusts the audio processing parameters according to the preset relationship, solves the problem of poor audio effects in the existing technology, realizes efficient audio processing, and reduces hardware costs and capabilities Consumption.
Patent Information
- Application Number
- CN202310519450.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-09
- Publication Date
- 2024-11-12
AI Technical Summary
When collecting audio, existing monitoring devices find it difficult to adjust audio processing parameters according to different scenes and sound source distances, resulting in poor audio effects and increasing hardware detection devices increase cost and power consumption.
The image of the object and the object shadow is collected by the image collector, the distance and orientation of the object and the image collector are determined, and the audio processing parameters are determined according to the preset relationship, and the audio processing parameters are adaptively adjusted to improve the audio effect.
The audio processing parameters can be accurately determined without additional hardware devices, improve the loudness and restore of the audio, reduce sound breaking distortion, and improve the audio processing effect and efficiency.
Smart Images

Figure CN118942472A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of audio processing technology, and in particular to an audio processing method, device, electronic device and medium. Background Art
[0002] Surveillance equipment is increasingly used in all walks of life. In many application scenarios, surveillance equipment needs to capture not only images but also audio. With the development of technology, the quality requirements for the collected audio are also increasing. The collected audio must not only have a loudness that meets the auditory needs, but also meet the requirements of high restoration and reduced distortion.
[0003] At present, the collected audio is often processed by algorithms. The audio effects collected in different scenarios are often different. If a single audio processing parameter is used to process the audio, it is difficult to achieve the best effect after processing all the collected audio. In addition, if the audio processing parameters are adjusted according to the distance of the sound source in the scene, additional hardware detection devices are required, which increases costs and power consumption. Summary of the invention
[0004] The present application provides an audio processing method, device, electronic device and medium, so as to detect the distance and direction between an object and an image collector without adding additional hardware devices, and adaptively determine target audio processing parameters.
[0005] According to one aspect of the present application, there is provided an audio processing method, the method comprising:
[0006] Acquire an image including an object and a shadow of the object by an image collector, and determine a target distance between the object and the image collector and a target orientation of the object relative to the image collector according to the image;
[0007] Determine the target audio processing parameter according to the target distance, the target orientation and a preset association relationship; wherein the preset association relationship is an association relationship between the distance, the orientation and the audio processing parameter;
[0008] The collected audio of the object is processed according to the target audio processing parameter.
[0009] According to another aspect of the present application, there is provided an audio processing device, the device comprising:
[0010] A recognition module, used for acquiring an image containing an object and a shadow of the object through an image collector, and determining a target distance between the object and the image collector, and a target orientation of the object relative to the image collector according to the image;
[0011] A target audio processing parameter determination module, used to determine the target audio processing parameter according to the target distance, the target orientation and a preset association relationship; wherein the preset association relationship is an association relationship between distance, orientation and audio processing parameters;
[0012] A processing module is used to process the collected audio of the object according to the target audio processing parameter.
[0013] According to another aspect of the present application, an electronic device is provided, the device comprising:
[0014] at least one processor; and
[0015] a memory communicatively connected to at least one processor; wherein,
[0016] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor can execute the audio processing method of any embodiment of the present application.
[0017] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the audio processing method of any embodiment of the present application when executed.
[0018] The technical solution of the embodiment of the present application is to acquire an image containing an object and its shadow through an image collector, and determine the target distance between the object and the image collector and the target orientation of the object relative to the image collector based on the image, so that the distance and orientation of the target relative to the image collector can be accurately determined without adding additional hardware devices; determine target audio processing parameters based on the target distance, the target orientation and a preset association relationship; wherein the preset association relationship is an association relationship between distance, orientation and audio processing parameters, so as to adaptively determine target audio processing parameters corresponding to the target distance and target orientation, and process the collected audio of the object based on the target audio processing parameters, so as to improve the audio effect.
[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0021] Figure 1 is a flowchart of an audio processing method provided according to Embodiment 1 of the present application;
[0022] Figure 2 is a flowchart of an audio processing method provided according to Embodiment 2 of the present application;
[0023] Figure 3 is a first schematic diagram of an object shadow provided according to the second embodiment of the present application;
[0024] Figure 4 is a second schematic diagram of an object shadow provided according to the second embodiment of the present application;
[0025] Figure 5 is a flowchart of an audio processing method provided according to Embodiment 3 of the present application;
[0026] Figure 6 This is a schematic diagram of a human silhouette triangle provided according to Embodiment 3 of the present application;
[0027] Figure 7 is a structural diagram of an audio processing device provided according to Embodiment 4 of the present application;
[0028] Figure 8 It is a structural diagram of an electronic device provided in Example 5 of the present application. DETAILED DESCRIPTION
[0029] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.
[0030] It should be noted that the terms "first", "second", "third", "fourth", "actual", "preset", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0031] Embodiment 1
[0032] Figure 1 This is a flowchart of an audio processing method provided in the first embodiment of the present application. The embodiment of the present application can be applied to the case of processing collected audio. Typically, it can be applied to the case of processing the audio of the monitored object in a monitoring scenario. The method can be executed by an audio processing device, which can be implemented in the form of hardware and / or software, and the audio processing device can be configured in an electronic device. Figure 1 As shown, the method includes:
[0033] S110, acquiring an image including an object and its shadow through an image collector, and determining a target distance between the object and the image collector, and a target orientation of the object relative to the image collector according to the image.
[0034] Among them, the image collector can be an image collector arranged above the object, such as a monitoring device in daily application scenarios. The object can be any object. In order to save energy consumption, the object here can be an object that can emit audio. The embodiment of the present application only processes the object that can emit audio according to this scheme, and does not process the object that cannot emit audio. The shadow of the object can be a shadow produced by any light source irradiating the object. The light source can be the sun, street lights, indoor lighting, flash lights of the image collector, etc., which are not limited here. In this scheme, the object shadow referenced when analyzing the target orientation of the object relative to the image collector must be the same light source irradiating the object. The target distance can be the straight-line distance from the object to the image collector. The orientation of the object relative to the image collector can include forward, backward and sideways, etc. Forward means that the front of the object faces the image collector, that is, the side of the object that emits audio faces the image collector. Backward means that the back of the object faces the image collector, that is, the opposite side of the side of the object that emits audio faces the image collector. Sideways means that the side of the object faces the image collector, that is, the side facing the image collector is neither the front nor the back.
[0035] Exemplarily, a light source in an environment shines on an object to generate a shadow of the object. The image collector can collect images containing the object and the shadow of the object. The image is recognized and analyzed to determine the target distance between the object in the image and the image collector, as well as the target orientation of the object relative to the image collector. In this solution, if the image collector is an intelligent image collector, the above process can be performed by the image collector. If the image collector is not an intelligent image collector, the image collector can send the image containing the object and the shadow of the object to a connected processor, and the processor can perform the above process.
[0036] In an embodiment of the present application, the method also includes: if the image captured by the image collector does not include the object shadow, controlling the image collector to turn on a flash; if the image captured by the image collector still does not include the object shadow, adjusting the brightness of the flash until the image captured by the image collector includes the object shadow.
[0037] Exemplarily, if there is a light source in the environment, the light source shines on the object to form the shadow of the object, and the image collector can collect an image containing the object and the shadow of the object. If there is no light source in the environment, there is no shadow of the object, and the image of the image collector does not include the shadow of the object, it is necessary to control the image collector to turn on the flash, so that the flash shines on the object to form the shadow of the object, and the image of the object and the shadow of the object is collected by the image collector. If the image of the image collector still does not include the shadow of the object, it may be that the light intensity of the flash is weak, resulting in the shadow of the object not being obvious. The brightness of the flash can be adjusted to make the shadow of the object more obvious, so that the image collected by the image collector contains the shadow of the object. The above scheme controls the flash of the image collector to turn on when there is no ambient light source and the shadow of the object does not exist, thereby ensuring that the image collected by the image collector contains the shadow of the object, so as to ensure the normal execution of the subsequent scheme, and avoid the problem that the subsequent process cannot be executed due to the absence of the shadow of the object due to the absence of the ambient light source. In addition, the absence of the object's shadow may also be due to the fact that the ambient light source is directly above the shadow, or the object's shadow is blocked by the object and is difficult to be captured by the image collector, or the illumination angles of multiple ambient light sources are opposite, resulting in the cancellation of the illumination intensity and the failure to form the object's shadow. In this case, the flash located at a specific position can be controlled to turn on according to the position of the object, for example, the flash located in the vertical direction of the image acquisition direction can be controlled to turn on to ensure that the flash is irradiated on the object to form the object's shadow. In addition, the brightness of the ambient light source can be controlled. When the illumination intensity of the ambient light sources cancels out, the illumination intensity of a part of the ambient light sources can be increased, and the illumination intensity of the ambient light sources in the opposite direction can be reduced, so that the synthesized light can be irradiated on the object to form the object's shadow. In addition, the light intensity of the ambient light source in a certain direction can be weakened or increased according to the position of the object, so that the object's shadow is projected at a position that can be captured by the image collector.
[0038] S120. Determine a target audio processing parameter according to the target distance, the target orientation, and a preset association relationship; wherein the preset association relationship is an association relationship between distance, orientation, and audio processing parameters.
[0039] Among them, the audio processing parameters may include AGC (Automatic Gain Control) parameters, ANR (Acive Noise Reduction) parameters, AEC (Auto echo cancellation) parameters, EQ (Equaliser) parameters, BF (Beamforming) parameters, etc. Exemplarily, a preset association relationship between distance, orientation and audio processing parameters may be predetermined, and the audio processing parameters are audio processing parameters that achieve the best effect after processing the audio of the collected object when the distance and orientation are known and fixed. After determining the target distance between the object and the image collector, and the target orientation of the object relative to the image collector, the audio processing parameters corresponding to the target distance and the target orientation are determined from the preset association relationship, and the audio processing parameters are used as the target audio processing parameters.
[0040] In an embodiment of the present application, the process of determining a preset association relationship includes: obtaining audio processing parameters corresponding to an object of a preset height at different distances and in different orientations relative to the image collector; the audio processing parameters are audio processing parameters corresponding to when the audio effect of the object is optimal; and establishing an association relationship among distance, orientation, and audio processing parameters as a preset association relationship.
[0041] Exemplarily, an object at a preset height can be pre-controlled to emit audio of a preset loudness and a preset frequency. The object is placed at different distances relative to the image collector, and the object is controlled to be in different directions relative to the image collector when it is at different distances. Under different distances and different directions, the audio of the object is collected by the microphone on the image collector, and the collected audio is processed by the audio processing parameters. When the processed audio reaches the best audio effect, the audio processing parameters are used as the best audio processing parameters, and the relationship between the distance, direction and audio processing parameters of the object relative to the image collector is established. Similarly, for each distance and each direction, the above process is performed to establish a relationship between each distance, each direction and audio processing parameters. Through the above scheme, the best audio processing parameters corresponding to each distance and direction can be clearly defined, which is convenient for the subsequent direct determination of the target audio processing parameters based on the preset association relationship to process the collected audio, improve the audio processing effect, and quickly adjust the audio to the best effect, thereby improving the adjustment efficiency.
[0042] S130: Process the collected audio of the object according to the target audio processing parameter.
[0043] Among them, the microphone for collecting target audio can be at the same point as the image collector. When the object is at a target distance and a target orientation relative to the image collector, the audio of the object is collected by the microphone. According to the target audio processing parameters, the collected audio is processed to improve the audio effect. Since the target audio processing parameters are the optimal audio processing parameters corresponding to the target distance and the target orientation determined according to the preset association relationship, the collected audio is processed according to the target audio processing parameters, and the collected audio can be quickly adjusted to the best auditory effect, thereby improving the audio display effect while improving the audio adjustment efficiency.
[0044] The technical solution of the embodiment of the present application is to acquire an image containing an object and its shadow through an image collector, and determine the target distance between the object and the image collector and the target orientation of the object relative to the image collector based on the image, so that the distance and orientation of the target relative to the image collector can be accurately determined without adding additional hardware devices; determine target audio processing parameters based on the target distance, the target orientation and a preset association relationship; wherein the preset association relationship is an association relationship between distance, orientation and audio processing parameters, so as to adaptively determine target audio processing parameters corresponding to the target distance and target orientation, and process the collected audio of the object based on the target audio processing parameters, so as to improve the audio effect.
[0045] Embodiment 2
[0046] Figure 2 This is a flowchart of an audio processing method provided in the second embodiment of the present application. The present embodiment is optimized based on the above embodiment. For solutions not described in detail in the present embodiment, please refer to the above embodiment. Figure 2 As shown, the method of the embodiment of the present application specifically includes the following steps:
[0047] S210: Capture an image including an object and its shadow through an image collector.
[0048] S220: Determine a target position of the object in the image and a target pixel range occupied by the object in the image.
[0049] The target position may be the pixel position of the object in the image. For example, the center of the object is located at the pixel of the nth row and nth column in the image. An object is generally not a point but has a certain volume, so the object generally occupies a certain pixel range in the image. For example, if the object occupies the pixel range between the third row to the seventh row and the second column to the fifth column in the image, the pixel range between the third row to the seventh row and the second column to the fifth column is taken as the target pixel range. In this scheme, as long as a pixel includes at least part of the image of the object, the pixel belongs to the target pixel range.
[0050] S230: Determine a target distance between the object and the image collector according to the target position and the target pixel point range and based on a preset corresponding relationship corresponding to the field of view range of the image collector.
[0051] The preset corresponding relationship is the corresponding relationship between the position of the object in the image, the range of pixels occupied by the object in the image, and the distance of the object relative to the image collector.
[0052] Exemplarily, the preset correspondence between the position of the object in the image, the range of pixels occupied by the object in the image, and the distance of the object relative to the image collector under different fields of view of the image collector can be predetermined. Specifically, for an image collector with a field of view, the image collector is used to collect images of an object located at a preset distance. Determine the position of the object in the image and the range of pixels occupied by the object in the collected image. Establish a preset correspondence between the position of the object in the image, the range of pixels occupied by the object in the image, and the distance under the field of view.
[0053] After determining the target position and the target pixel range, the distance corresponding to the target position and the target pixel range is determined based on the preset correspondence corresponding to the field of view range of the image collector, that is, the target distance between the object and the image collector. Thus, by analyzing the image, the distance between the object and the image collector can be determined quickly and efficiently without the need for additional hardware devices, and the distance between the object and the image collector can be accurately determined.
[0054] In an embodiment of the present application, for a preset number of objects with the same deflection angle and different distances relative to the image collector, the image collector can be used to collect images of the preset number of objects in advance, determine the position of the object in the image, and determine the actual distance between the object and the image collector. For a preset number of objects with the same distance and different deflection angles relative to the image collector, the image collector can be used to collect images of the preset number of objects in advance, determine the position of the object in the image, and determine the actual distance between the object and the image collector. Establish an association between the position of each object in the image and the actual distance between the object and the image collector. After the image collector is used to collect the image of the object to obtain the image of the object, the target position of the object in the image is determined, and the target distance between the object and the image collector is determined based on the association and the target position.
[0055] S240: Determine the length and direction of the object shadow according to the image.
[0056] In this solution, the length of the object shadow can be the pixel range occupied by the object in the image, or it can be the actual length, which is not limited here. For example, similar to the solution in S230, the relationship between the field of view range and the resolution of the image can be predetermined, and a mapping relationship between the pixel range in the image and the actual range can be established. Determine the length of the object shadow in the image, that is, the number of pixels occupied by the object shadow in the image, and then determine the actual length corresponding to the pixel based on the mapping relationship.
[0057] The direction of the object shadow may be the direction of the object shadow in the image, such as the horizontal direction, the vertical direction, the direction with an angle of 45 degrees to the horizontal direction, etc. The direction of the object shadow may also be the actual direction of the object shadow, such as the due south direction, the due north direction, the southwest direction, etc.
[0058] S250: Determine a target orientation of the object relative to the image collector according to the length of the object's shadow and the direction of the object's shadow.
[0059] For example, as the object moves relative to the image collector, the length and direction of the object's shadow may change, and the target orientation of the object relative to the image collector may be determined based on the changes.
[0060] In an embodiment of the present application, the target orientation of the object relative to the image collector is determined according to the length of the object shadow and the direction of the object shadow, including: splitting the object shadow into a first component of a first preset direction and a second component of a second preset direction according to the length of the object shadow and the direction of the object shadow; wherein the first preset direction is the direction of the projection vector of the vector connecting the object and the image collector on the ground; the second preset direction is a direction perpendicular to the first preset direction; and determining the target orientation of the object relative to the image collector according to the direction of the object shadow, the first component, and the second component.
[0061] For example, Figure 3 As shown, assuming that the image collector and the light source are located on the same side, the object shadows of the object located at position 1, position 2, and position 3 are object shadow 1, object shadow 2, and object shadow 3, respectively. Figure 3 The x-axis, y-axis and z-axis are perpendicular to each other, and the image collector and the light source are located in the plane formed by the x-axis and the z-axis. The projection vector of the vector connecting the object and the image collector on the ground can be the x-axis direction, and the second preset direction can be the y-axis direction. The object shadow can be split into a first component in the x-axis direction and a second component in the y-axis direction. Figure 3 The object shadow 1, object shadow 2 and object shadow 3 in the figure are all located on the x-axis, so their second components on the y-axis are all 0. The length of the x-axis component of object shadow 1 is the length of object shadow 1, and the same applies to other object shadows. If the change of the first component of the object shadow is getting longer and longer, it can be determined that the object position changes are position 1-position 2-position 3, that is, the direction relative to the image collector is back. If the change of the first component of the object shadow is getting shorter and shorter, it can be determined that the object position changes are position 3-position 2-position 1, that is, the direction relative to the image collector is forward.
[0062] For example, Figure 4 As shown, if the image collector and the light source are located in the plane formed by the z-axis and the y-axis, the projection vector of the vector connecting the object and the image collector on the ground can be the y-axis direction, and the second preset direction can be the x-axis direction. The object shadow can be split into a first component of the y-axis and a second component of the x-axis. If the second component of the object shadow changes from -0 along the negative axis of the x-axis to the positive axis of the x-axis, and the first component is along the negative axis of the y-axis and the length does not change, it is determined that the object is lateral relative to the image collector.
[0063] The above are just two examples of shadow changes of objects. There may also be shadow changes under other positional relationships between the image collector, light source and object, which are not listed here one by one.
[0064] S260. Determine a target audio processing parameter according to the target distance, the target orientation, and a preset association relationship; wherein the preset association relationship is an association relationship between distance, orientation, and audio processing parameters.
[0065] S270: Process the collected audio of the object according to the target audio processing parameter.
[0066] The embodiment of the present application provides an audio processing method, which determines the target position of the object in the image and the target pixel range occupied by the object in the image, and determines the target distance between the object and the image collector based on the preset correspondence corresponding to the field of view of the image collector according to the target position and the target pixel range. Determine the length of the object's shadow and the direction of the object's shadow according to the image; determine the target orientation of the object relative to the image collector according to the length of the object's shadow and the direction of the object's shadow, so as to accurately determine the target distance and target orientation of the object relative to the image collector without adding other hardware devices, so as to adaptively determine the optimal target audio processing parameters and process the audio to achieve the best audio effect.
[0067] Embodiment 3
[0068] Figure 5 This is a flowchart of an audio processing method provided in Embodiment 3 of the present application. The present embodiment is optimized based on the above embodiment. For solutions not described in detail in the present embodiment, see the above embodiment. Figure 5 As shown, the method of the embodiment of the present application specifically includes the following steps:
[0069] S310: Capture an image including an object and its shadow through an image collector, and determine a target distance between the object and the image collector according to the image.
[0070] S320. Determine a human shadow triangle according to the object and the object shadow in the image; wherein three vertices of the human shadow triangle are respectively the highest point of the object, the farthest point of the object shadow from the object, and the intersection point of the object and the object shadow.
[0071] For example, Figure 6 As shown, the highest point, the lowest point of the object and the point where the object's shadow is farthest from the object are connected to obtain a shadow triangle.
[0072] S330: Determine the target orientation of the object relative to the image collector according to the inner angle of the human shadow triangle.
[0073] For example, Figure 6 As shown, when the object is at position 1, position 2, and position 3, the corresponding internal angles of the human shadow triangle include a. If the internal angle a of the human shadow triangle changes to gradually increase, and the position change of the object is determined to be position 1-position 2-position 3, then it can be determined that the target orientation of the object relative to the image collector is back. If the internal angle a of the human shadow triangle changes to gradually decrease, and the position change of the object is determined to be position 3-position 2-position 1, then it can be determined that the target orientation of the object relative to the image collector is forward. The internal angle of the human shadow triangle can also be determined as another internal angle that is complementary to a.
[0074] S340, determining a target audio processing parameter according to the target distance, the target orientation, and a preset association relationship; wherein the preset association relationship is an association relationship between distance, orientation, and audio processing parameters.
[0075] S350: Process the collected audio of the object according to the target audio processing parameter.
[0076] An embodiment of the present application provides an audio processing method, which determines a human shadow triangle according to the object and the object shadow in the image; wherein the three vertices of the human shadow triangle are respectively the highest point of the object, the farthest point of the object shadow from the object, and the intersection point of the object and the object shadow; based on the inner angle of the human shadow triangle, the target orientation of the object relative to the image collector is determined, so that without adding other hardware devices, the target distance and target orientation of the object relative to the image collector can be accurately determined, so as to adaptively determine the optimal target audio processing parameters and process the audio to achieve the best audio effect.
[0077] Embodiment 4
[0078] Figure 7 This is a structural diagram of an audio processing device provided in Embodiment 4 of the present application. The device can execute the audio processing method provided in any embodiment of the present application and has the corresponding functional modules and beneficial effects of the execution method. Figure 7 As shown, the device comprises:
[0079] The recognition module 410 is used to collect an image containing an object and a shadow of the object through an image collector, and determine a target distance between the object and the image collector, and a target orientation of the object relative to the image collector according to the image;
[0080] The target audio processing parameter determination module 420 is used to determine the target audio processing parameter according to the target distance, the target orientation and a preset association relationship; wherein the preset association relationship is an association relationship between the distance, the orientation and the audio processing parameter;
[0081] The processing module 430 is used to process the collected audio of the object according to the target audio processing parameter.
[0082] In the embodiment of the present application, the identification module 410 includes:
[0083] A target pixel range determination unit, used to determine a target position of the object in the image, and a target pixel range occupied by the object in the image;
[0084] A target distance determination unit is used to determine the target distance between the object and the image collector according to the target position and the target pixel point range, based on a preset correspondence corresponding to the field of view range of the image collector; wherein the preset correspondence is the correspondence between the position of the object in the image, the pixel point range occupied by the object in the image, and the distance of the object relative to the image collector.
[0085] In the embodiment of the present application, the identification module 410 includes:
[0086] a length and direction determining unit, configured to determine the length of the object shadow and the direction of the object shadow according to the image;
[0087] The target orientation determining unit is used to determine the target orientation of the object relative to the image collector according to the length of the object's shadow and the direction of the object's shadow.
[0088] In an embodiment of the present application, the target direction determination unit includes:
[0089] a component determination subunit, configured to split the object shadow into a first component in a first preset direction and a second component in a second preset direction according to the length of the object shadow and the direction of the object shadow; wherein the first preset direction is the direction of a projection vector of a vector connecting the object and the image collector on the ground; and the second preset direction is a direction perpendicular to the first preset direction;
[0090] The orientation determination subunit is used to determine the target orientation of the object relative to the image collector according to the direction of the shadow of the object, the first component and the second component.
[0091] In the embodiment of the present application, the identification module 410 includes:
[0092] A human shadow triangle determining unit, configured to determine a human shadow triangle according to the object and the object shadow in the image; wherein three vertices of the human shadow triangle are respectively the highest point of the object, the farthest point of the object shadow from the object, and the intersection point of the object and the object shadow;
[0093] An inner angle determination unit is used to determine the target orientation of the object relative to the image collector according to the inner angle of the human shadow triangle.
[0094] In the embodiment of the present application, the device further includes:
[0095] An audio processing parameter acquisition module, used to acquire audio processing parameters corresponding to different distances and different orientations of an object at a preset height relative to the image collector; the audio processing parameters are audio processing parameters corresponding to the best audio effect of the object;
[0096] The preset association relationship determination module is used to establish an association relationship among distance, orientation and audio processing parameters as a preset association relationship.
[0097] In the embodiment of the present application, the device further includes:
[0098] A flash control module, used for controlling the image collector to turn on the flash if the image collected by the image collector does not include the shadow of the object;
[0099] The brightness adjustment module is used to adjust the brightness of the flashlight if the image collected by the image collector still does not include the object shadow, until the image collected by the image collector includes the object shadow.
[0100] An audio processing device provided in an embodiment of the present application can execute an audio processing method provided in any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.
[0101] Embodiment 5
[0102] Figure 8 It is a structural diagram of an electronic device provided in Example 5 of the present application. Figure 8 A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.
[0103] like Figure 8As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0104] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0105] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as an audio processing method.
[0106] In some embodiments, the audio processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the audio processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the audio processing method in any other appropriate manner (e.g., by means of firmware).
[0107] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0108] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable audio processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.
[0109] In the context of the present application, a computer readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device or equipment. A computer readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium may be a machine readable signal medium. A more specific example of a machine readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0110] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0111] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0112] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0113] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the information expected by the technical solution of this application can be achieved, and this document is not limited here.
[0114] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.
Claims
1. An audio processing method, characterized in that: The method comprises: Acquire an image including an object and a shadow of the object by an image collector, and determine a target distance between the object and the image collector and a target orientation of the object relative to the image collector according to the image; Determine the target audio processing parameter according to the target distance, the target orientation and a preset association relationship; wherein the preset association relationship is an association relationship between the distance, the orientation and the audio processing parameter; The collected audio of the object is processed according to the target audio processing parameter.
2. The method according to claim 1, characterized in that Determining a target distance between the object and the image collector according to the image includes: Determine a target position of the object in the image, and a target pixel range occupied by the object in the image; According to the target position and the target pixel range, the target distance between the object and the image collector is determined based on a preset correspondence corresponding to the field of view range of the image collector; wherein the preset correspondence is a correspondence between the position of the object in the image, the pixel range occupied by the object in the image, and the distance of the object relative to the image collector.
3. The method according to claim 1, characterized in that Determining a target orientation of the object relative to the image collector according to the image includes: determining the length of the object's shadow and the direction of the object's shadow according to the image; The target orientation of the object relative to the image collector is determined according to the length of the object's shadow and the direction of the object's shadow.
4. The method according to claim 3, characterized in that Determining the target orientation of the object relative to the image collector according to the length of the object shadow and the direction of the object shadow includes: According to the length of the object shadow and the direction of the object shadow, the object shadow is split into a first component in a first preset direction and a second component in a second preset direction; wherein the first preset direction is the direction of a projection vector of a vector connecting the object and the image collector on the ground; and the second preset direction is a direction perpendicular to the first preset direction; A target orientation of the object relative to the image collector is determined according to the direction of the object's shadow, the first component, and the second component.
5. The method according to claim 1, characterized in that Determining a target orientation of the object relative to the image collector according to the image includes: Determine a human shadow triangle according to the object and the object shadow in the image; wherein three vertices of the human shadow triangle are respectively the highest point of the object, the farthest point of the object shadow from the object, and the intersection point of the object and the object shadow; The target orientation of the object relative to the image collector is determined according to the inner angle of the human shadow triangle.
6. The method according to claim 1, characterized in that The process of determining the preset association relationship includes: Acquire the audio processing parameters corresponding to different distances and different orientations of an object at a preset height relative to the image collector; the audio processing parameters are the audio processing parameters corresponding to the best audio effect of the object; An association relationship among the distance, the direction, and the audio processing parameter is established as a preset association relationship.
7. The method according to claim 1, characterized in that The method further comprises: If the image acquired by the image collector does not include the shadow of the object, controlling the image collector to turn on a flash light; If the image collected by the image collector still does not include the object shadow, the brightness of the flashlight is adjusted until the image collected by the image collector includes the object shadow.
8. An audio processing device, characterized in that: The device comprises: A recognition module, used for acquiring an image containing an object and a shadow of the object through an image collector, and determining a target distance between the object and the image collector, and a target orientation of the object relative to the image collector according to the image; A target audio processing parameter determination module, used to determine the target audio processing parameter according to the target distance, the target orientation and a preset association relationship; wherein the preset association relationship is an association relationship between distance, orientation and audio processing parameters; A processing module is used to process the collected audio of the object according to the target audio processing parameter.
9. An electronic device, characterized in that: The device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the audio processing method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the audio processing method according to any one of claims 1 to 7 when executed.
Citation Information
Cited By
Visual processing-based record audio identification and playing method, device and system
CN122593734A
A record audio identification and playing method, device and system based on visual processing
CN122593734B
