Speech enhancement method, device and system

By wearing sound pickup equipment in the 4S store and determining the location information of reception personnel and customers in combination with surveillance video, the problem of poor voice enhancement effect in noisy environments in the 4S store is solved, and high-quality voice enhancement and the accuracy of reception quality inspection are achieved.

CN114495959BActive Publication Date: 2024-11-29IFLYTEK CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111531077.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-14
Publication Date
2024-11-29
Estimated Expiration
2041-12-14

AI Technical Summary

Technical Problem

The noisy environment and many people in the 4S store lead to poor results in traditional voice enhancement algorithms, making it difficult to accurately conduct sales reception quality inspections.

Method used

The recording audio is obtained by the sound pickup device worn on the target receptionist, and the orientation information of the target receptionist and the customer is determined in combination with the monitoring video, the noise reduction parameters are determined, and the recording audio is reduced.

Benefits of technology

It realizes high-quality voice enhancement, improves the accuracy and reliability of reception quality inspection, and is suitable for reception quality inspection in complex scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114495959B_ABST
    Figure CN114495959B_ABST
Patent Text Reader

Abstract

A method, device and system for speech enhancement, the method comprising: obtaining audio recording of a process in which a target receptionist receives a target customer, wherein the audio recording is picked up by a sound pickup device worn on the target receptionist; obtaining a surveillance video of the same time period as the audio recording, and determining the target receptionist and other personnel from the surveillance video; determining the target customer from the other personnel based on the target receptionist; determining the position information of the target customer relative to the target receptionist during the time period, and determining noise reduction parameters based on the position information; and performing noise reduction processing on the audio recording based on the noise reduction parameters. The present application places a sound pickup device on one of the two objects involved in the reception process, locks the object in combination with the surveillance video, and then locates the other object, and finally obtains the relative position between the two for noise reduction, which can achieve a high-quality noise reduction effect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of speech enhancement technology, and more specifically to a speech enhancement method, device and system. Background Art

[0002] When customers go to 4S stores to buy cars, 4S store staff often arrange one-on-one reception. In order to test the professionalism and service attitude of store sales during the reception process, and to better understand customers' car purchase intentions and concerns, 4S store reception quality inspection came into being. The so-called store quality inspection means that sales record the process of receiving customers, and then judge the professionalism and service attitude of sales reception through transcription and semantic extraction, such as whether polite language is used, whether vehicle parameters and store activities are introduced completely and correctly. In addition, through the analysis of customers, their car purchase intentions, car purchase concerns, etc. are understood, and electronic work orders are generated to facilitate subsequent follow-up visits to customers and the pursuit of car purchases. However, the environment in 4S stores is very noisy, resulting in very poor transcription of recordings, thereby reducing the accuracy of quality inspection.

[0003] The current quality inspection system generally performs conventional noise reduction processing on the recorded audio to enhance the recording. However, the environment of 4S stores is very noisy, and there are many people viewing cars and receiving guests at the same time, so the recording often contains a lot of interference from other people's voices. In addition, during the reception process, the positions of sales and customers are not fixed, which will cause a large change in the recording volume. Traditional speech enhancement algorithms have poor effects on 4S store scenarios. In other business scenarios, there are methods that locate the speaker based on face recognition, and then pick up the voice in the located direction through a microphone array to achieve the effect of noise reduction. However, the 4S store is very large, with many people and frequent movement. Fixed cameras are affected by line of sight obstruction and cannot effectively obtain image information of specific people (such as a sales consultant who is receiving guests) in real time. Summary of the invention

[0004] The present application is proposed to solve at least one of the above problems. According to one aspect of the present application, a speech enhancement method is provided, the method comprising: obtaining a recording audio of a process in which a target receptionist receives a target customer, wherein the recording audio is picked up by a sound pickup device worn on the target receptionist; obtaining a surveillance video of the same time period as the recording audio, and determining the target receptionist and other personnel from the surveillance video; determining the target customer from the other personnel based on the target receptionist; determining the position information of the target customer relative to the target receptionist within the time period, and determining noise reduction parameters based on the position information; and performing noise reduction processing on the recording audio based on the noise reduction parameters.

[0005] In one embodiment of the present application, the method also includes: obtaining the location information of the target reception staff during the process of the target reception staff receiving the target customer; determining the target reception staff and other personnel from the surveillance video includes: obtaining the location information of each person included in the surveillance video during the time period; comparing the location information of the target reception staff with the location information of each person during the time period to determine the target reception staff from the various personnel, and personnel other than the target reception staff are other personnel.

[0006] In one embodiment of the present application, comparing the location information of the target reception staff with the location information of each staff in the time period to determine the target reception staff from the various staff includes: acquiring the movement trajectory of the target reception staff in the process of receiving the target customer based on the location information of the target reception staff; acquiring the movement trajectory of each staff in the time period based on the location information of each staff in the time period; matching the movement trajectory of the target reception staff in the process of receiving the target customer with the movement trajectory of each staff in the time period to determine the target reception staff from the various staff.

[0007] In one embodiment of the present application, the recorded audio includes identification information of the target receptionist, and determining the target receptionist and other personnel from the surveillance video includes: obtaining features corresponding to the identification information from a pre-stored feature library, and determining the target receptionist from the surveillance video based on the features, and personnel other than the target receptionist are other personnel.

[0008] In one embodiment of the present application, determining the target customer from the other personnel based on the target reception staff includes: obtaining dynamic distance information between the target reception staff and the surrounding personnel in the surveillance video, and determining the target customer from the other personnel based on the dynamic distance information.

[0009] In one embodiment of the present application, the dynamic distance information between the target reception staff and the surrounding persons in the surveillance video and the target customer is determined from the other personnel based on the dynamic distance information, including: obtaining the dynamic change of the distance between the target reception staff and the surrounding persons in the surveillance video; and determining the person with the smallest dynamic change of the distance to the target reception staff as the target customer.

[0010] In one embodiment of the present application, the determining of the position information of the target customer relative to the target receptionist during the time period is also based on the facial orientation information of the target receptionist during the time period.

[0011] In one embodiment of the present application, determining the noise reduction parameters according to the orientation information includes: determining the parameters of a beamforming algorithm according to the orientation information.

[0012] In one embodiment of the present application, the position information within the time period varies with time, and the parameters of the beamforming algorithm also vary with time.

[0013] In one embodiment of the present application, the relative position between the sound pickup device and the target receptionist is fixed, and the method further includes: obtaining the orientation information of the sound pickup device relative to the target receptionist; and performing noise reduction processing on the recorded audio based on the orientation information of the sound pickup device relative to the target receptionist.

[0014] In one embodiment of the present application, the sound pickup device has a positioning function to obtain the location information of the target receptionist.

[0015] In one embodiment of the present application, obtaining the location information of each person included in the surveillance video during the time period includes: constructing a three-dimensional person model of each person included in the surveillance video based on the positions, angles and captured image ranges of multiple image acquisition devices that capture the surveillance video; and identifying and tracking the position of each of the three-dimensional person models to obtain the location information of each person during the time period.

[0016] In one embodiment of the present application, the recorded audio and the monitored video are uploaded to a background server in real time so that the background server executes the voice enhancement method.

[0017] In one embodiment of the present application, the recorded audio after noise reduction processing is transcribed and semantically extracted to obtain the reception quality inspection result of the target reception staff, so as to realize the reception quality inspection of the target reception staff.

[0018] According to another aspect of the present application, a speech enhancement method is provided, the method comprising: obtaining recorded audio of a process in which a target receptionist receives a target customer, wherein the recorded audio is picked up by a sound pickup device worn on the target customer; obtaining a surveillance video of the same time period as the recorded audio, and determining the target customer and other personnel from the surveillance video; determining the target receptionist from the other personnel based on the target customer; determining the position information of the target receptionist relative to the target customer within the time period, and determining noise reduction parameters based on the position information; and performing noise reduction processing on the recorded audio based on the noise reduction parameters.

[0019] According to another aspect of the present application, a speech enhancement device is provided, the device comprising a memory and a processor, the memory storing a computer program executed by the processor, and when the computer program is executed by the processor, the processor executes the above-mentioned speech enhancement method.

[0020] According to another aspect of the present application, a speech enhancement system is provided, the system comprising an image acquisition device, a sound pickup device and the above-mentioned speech enhancement device, wherein the image acquisition device is used to acquire surveillance video of a reception place, the sound pickup device is used to acquire recorded audio of a process in which a receptionist receives a customer, and the speech enhancement device is used to perform noise reduction on the recorded audio based on the surveillance video and the recorded audio.

[0021] According to the speech enhancement method, device and system of the embodiment of the present application, the sound pickup device is placed on one of the two objects involved in the reception process, and the object is locked in combination with the surveillance video and then the other object is located, and finally the relative position between the two is obtained for noise reduction, which can achieve high-quality noise reduction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0023] Figure 1 A schematic flow chart of a speech enhancement method according to an embodiment of the present application is shown.

[0024] Figure 2 A schematic flow chart of a speech enhancement method according to another embodiment of the present application is shown.

[0025] Figure 3A schematic structural block diagram of a speech enhancement device according to an embodiment of the present application is shown.

[0026] Figure 4 A schematic structural block diagram of a speech enhancement system according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present application more obvious, the example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application described in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present application.

[0028] Figure 1 FIG. 1 is a schematic flow chart of a speech enhancement method 100 according to an embodiment of the present application. Figure 1 As shown, the speech enhancement method 100 may include the following steps:

[0029] In step S110, the audio recording of the process in which the target receptionist receives the target customer is obtained, wherein the audio recording is picked up by a sound pickup device worn on the target receptionist.

[0030] In step S120, a surveillance video of the same time period as the recorded audio is obtained, and the target receptionist and other personnel are identified from the surveillance video.

[0031] In step S130, target customers are determined from other personnel according to the target reception personnel.

[0032] In step S140, the position information of the target customer relative to the target receptionist during the time period is determined, and the noise reduction parameters are determined according to the position information.

[0033] In step S150, noise reduction processing is performed on the recorded audio according to the noise reduction parameters.

[0034] In an embodiment of the present application, there are many people moving around in a 4S store or other similar scenes, including receptionists and customers. For each customer, a corresponding receptionist (such as a salesperson in a 4S store or other similar receptionist) will be assigned. For each receptionist, the recorded audio between him and the corresponding customer (the recorded audio of the receptionist during the reception of his customer) can be performed based on the voice enhancement method 100 provided in the present application to perform voice enhancement, so as to be used for reception quality inspection of the reception performed by the receptionist. Therefore, the target receptionist in the voice enhancement method 100 of the present application refers to any receptionist, and the target customer refers to the customer received by the receptionist.

[0035] In an embodiment of the present application, the target receptionist's reception process for the target customer is recorded by a sound pickup device worn on the target receptionist, so the sound pickup of the target receptionist can be directionally picked up by fixed beamforming, thereby obtaining a "clean" receptionist voice (the fixed beamforming algorithm is a conventional noise reduction algorithm, which will not be repeated here). Therefore, as long as the position of the target customer relative to the target receptionist during the entire reception process can be obtained, the parameters of the beamforming algorithm can be determined to perform noise reduction processing on the recorded audio, and directional pickup of the target customer's voice can be achieved, thereby obtaining a "clean" customer voice. In this way, both the "clean" receptionist voice and the "clean" customer voice can be obtained, and after such operations as transcription and semantic extraction, the reception quality inspection results of the target receptionist can be obtained. Since both the receptionist voice and the customer voice have been noise-reduced with high quality and become clean, the reception quality inspection results obtained based on them naturally also have high accuracy.

[0036] In an embodiment of the present application, a surveillance video is taken of a scene where many receptionists receive customers. In the surveillance video, different people in the video can be distinguished (such as based on face, height, body shape, etc.), and the position information, facial feature information, etc. of each person in the video at different times can be obtained. Based on this, when a section of recorded audio is to be voice enhanced, a surveillance video of the same period as the recorded audio can be obtained, and the position information of each person in the surveillance video during this period can be obtained. Then, based on the position information of the object in the recorded audio, that is, the target receptionist during the reception process (for example, obtained by a positioning device or by a sound pickup device with a positioning function worn on the target receptionist), the target receptionist can be locked (found) from the surveillance video. Alternatively, the recorded audio may include identification information of the target receptionist, and the identification information may correspond to the facial feature information in the pre-stored feature library. Based on this, face recognition can also be performed on the surveillance video to determine the target receptionist. After the target receptionist is determined, the surrounding personnel of the target receptionist can be found in the monitoring video. Since the target customer corresponding to the target receptionist is being received by the target receptionist, that is, the distance between the target customer and the target receptionist should be relatively fixed / stable during the entire reception process, while the distance between the remaining irrelevant surrounding personnel and the target receptionist should change randomly during the process, the target customer can be determined from the monitoring video based on the dynamic distance information between the surrounding personnel of the target receptionist and the target receptionist. In this way, in the monitoring video, the positions of the target receptionist and the target customer at different times have been determined, and the position of the target customer relative to the target receptionist at different times can be determined. Finally, as mentioned above, by obtaining the position of the target customer relative to the target receptionist during the entire reception process, the parameters of the beamforming algorithm (noise reduction parameters) can be determined to perform noise reduction processing on the recorded audio, thereby achieving directional pickup of the target customer's voice, thereby obtaining a "clean" customer voice, combined with the "clean" receptionist voice that can be directionally picked up by the sound pickup device worn by the target receptionist, to obtain a "clean" recorded audio after noise reduction processing. The reception quality inspection results obtained based on this will inevitably also have high accuracy.

[0037] Therefore, in general, according to the voice enhancement method of the embodiment of the present application, the sound pickup device worn on the target receptionist is used to record the reception process of the target receptionist with the target customer to obtain high-quality receptionist voice, and then based on the location information and / or facial information of the target receptionist during the reception process, combined with the surveillance video of the same period as the recorded audio, the target receptionist and the target customer are locked in turn from the surveillance video, and the relative position between the two is obtained to determine the noise reduction parameters. After the recorded audio is denoised by the noise reduction parameters, high-quality customer voice can be obtained. The high-quality receptionist voice and customer voice provide accuracy guarantee for the subsequent reception quality inspection.

[0038] In an embodiment of the present application, the location information of the target receptionist during the reception process can be obtained from a positioning device (such as a mobile phone, etc.) on the target receptionist, or it can be obtained from a sound pickup device worn by the target receptionist and recording the audio. In the latter example, the sound pickup device needs to have a positioning function. In this way, the location information of the target receptionist can be obtained while obtaining the audio recording from the sound pickup device, which is very convenient.

[0039] In an embodiment of the present application, obtaining the location information of each person included in the surveillance video during the period may include: constructing a three-dimensional person model of each person included in the surveillance video according to the position, angle and range of the captured image of one or more image acquisition devices that capture the surveillance video; and identifying and tracking the position of each three-dimensional person model to obtain the location information of each person during the period. In this embodiment, one or more cameras may be distributed in the scene (such as a 4S store), which collect surveillance videos in the scene, and the surveillance videos may be uploaded to the backend server. The backend server may construct a three-dimensional person model in the scene according to the position, angle, range of the captured image and other features of each camera, and obtain information such as the position and movement trajectory of each person.

[0040] In an embodiment of the present application, the location information of the target receptionist is compared with the location information of each person in the time period to determine the target receptionist from each person, which may include: obtaining the movement trajectory of the target receptionist in the process of receiving the target customer based on the location information of the target receptionist; obtaining the movement trajectory of each person in the time period based on the location information of each person in the time period; matching the movement trajectory of the target receptionist in the process of receiving the target customer with the movement trajectory of each person in the time period to determine the target receptionist from each person. In this embodiment, the movement trajectory of the target receptionist in a certain time period (the time period corresponding to the recorded audio) and each person in the monitoring video is obtained through the location information at different times, and the person can be quickly determined from the monitoring video by matching the movement trajectory. Among them, the movement trajectory of the target receptionist is obtained by the location information of the target receptionist obtained as described above, and the movement trajectory of each person in the monitoring video is obtained based on the monitoring video, as described above.

[0041] In an embodiment of the present application, obtaining dynamic distance information between the target receptionist and the surrounding personnel of the target receptionist in the surveillance video, and determining the target customer from each person based on the dynamic distance information, can include: obtaining the dynamic change of the distance between the target receptionist and the surrounding personnel of the target receptionist in the surveillance video; determining the person with the smallest dynamic change of the distance from the target receptionist as the target customer. As mentioned above, the distance of the target customer from the target receptionist should be relatively fixed / stable during the entire reception process, while the distance of the remaining irrelevant surrounding personnel from the target receptionist should change randomly during the process. Therefore, in this embodiment, the dynamic change of the distance between the surrounding personnel of the target receptionist and the target receptionist can be calculated, and the person with the smallest dynamic change of the distance from the target receptionist can be determined as the target customer.

[0042] In an embodiment of the present application, the position information of the target customer relative to the target receptionist during the time period can be determined based not only on the position information of the target customer and the target receptionist, but also on the facial orientation information of the target receptionist during the time period, which helps to more quickly and accurately determine the position information of the target customer relative to the target receptionist during the time period. In addition, the determined noise reduction parameters can be parameters of a beamforming algorithm to achieve directional pickup of the target customer's voice in the recorded audio. In the process of the target receptionist receiving the target customer, the above-mentioned position information can change over time, and accordingly, the parameters of the beamforming algorithm can also change over time to achieve directional pickup of the target customer's voice in the recorded audio.

[0043] In an embodiment of the present application, the recorded audio after noise reduction processing in step S160 can be transcribed and semantically extracted to obtain the reception quality inspection result of the target reception staff for the target customer, so as to realize the reception quality inspection of the target reception staff. Among them, transcription and semantic extraction are conventional methods of quality inspection, which will not be repeated here.

[0044] In the embodiment of the present application, the recorded audio and the monitored video obtained in steps S110 and S120 can be uploaded to the background server in real time, so that after the background server obtains the recorded audio at one end in real time, it can obtain the monitored video of the same period to lock the target person and execute the voice enhancement method of the present application. Alternatively, the recorded audio and the monitored video obtained in steps S110 and S120 can also be obtained in non-real time, that is, offline voice enhancement and reception quality inspection can also be performed.

[0045] Based on the above description, according to the voice enhancement method 100 of the embodiment of the present application, the sound pickup device worn on the target receptionist is used to record the reception process of the target receptionist to the target customer to obtain high-quality receptionist voice, and then combined with the surveillance video of the same period as the recorded audio, the target receptionist and the target customer are locked in turn from the surveillance video, and the relative position between the two is obtained to determine the noise reduction parameters. After the recorded audio is denoised by the noise reduction parameters, high-quality customer voice can be obtained. The high-quality receptionist voice and customer voice provide accuracy guarantee for the subsequent reception quality inspection, which is very suitable for reception quality inspection in various complex scenarios (such as 4S stores).

[0046] The above is an exemplary description of a speech enhancement method according to an embodiment of the present application. Figure 2 A speech enhancement method 200 according to another embodiment of the present application is described. Figure 2 As shown, the speech enhancement method 200 may include the following steps:

[0047] In step S210, the audio recording of the process in which the target receptionist receives the target customer is obtained, wherein the audio recording is picked up by a sound pickup device worn on the target customer.

[0048] In step S220, a surveillance video of the same time period as the recorded audio is obtained, and the target customer and other personnel are identified from the surveillance video.

[0049] In step S230, a target receptionist is determined from other personnel according to the target customer.

[0050] In step S240, the position information of the target receptionist relative to the target customer during the time period is determined, and the noise reduction parameters are determined according to the position information.

[0051] In step S250, noise reduction processing is performed on the recorded audio according to the noise reduction parameters.

[0052] The speech enhancement method 200 according to the embodiment of the present application is substantially similar to the speech enhancement method 100 described above, except that: the speech enhancement method 100 is to pick up the recorded audio by a sound pickup device worn on the target receptionist, and then first lock the target receptionist in the surveillance video in combination with the surveillance video, and then locate the target customer based on the target receptionist, and finally determine the relative position between the two to perform noise reduction; while the speech enhancement method 200 is to pick up the recorded audio by a sound pickup device worn on the target customer, and then first lock the target customer in the surveillance video in combination with the surveillance video, and then locate the target receptionist based on the target customer, and finally determine the relative position between the two to perform noise reduction. For the sake of brevity, only the differences between the speech enhancement methods 100 and 200 are described here. For their similarities, please refer to the previous text and will not be repeated here.

[0053] Based on the above description, according to the voice enhancement method 200 of the embodiment of the present application, the sound pickup device worn on the target customer is used to record the reception process of the target receptionist to the target customer to obtain high-quality customer voice, and then combined with the surveillance video of the same period as the recorded audio, the target customer and the target receptionist are locked in turn from the surveillance video, and the relative position between the two is obtained to determine the noise reduction parameters. After the recorded audio is denoised by the noise reduction parameters, high-quality customer voice can be obtained. The high-quality receptionist voice and customer voice provide accuracy guarantee for the subsequent reception quality inspection, which is very suitable for reception quality inspection in various complex scenarios (such as 4S stores).

[0054] In general, the voice enhancement methods 100 and 200 are to place the sound pickup device on one of the two objects involved in the reception process, lock the object in combination with the surveillance video, and then locate the other object, and finally obtain the relative position between the two for noise reduction. Therefore, the voice enhancement effects of the voice enhancement methods 100 and 200 are similar, and both can be used to obtain high-accuracy reception quality inspection results.

[0055] The above exemplary illustrates the speech enhancement method according to the embodiment of the present application. Figure 3 The speech enhancement device 300 provided according to another aspect of the present application is described. Figure 3 As shown, the speech enhancement device 300 may include a memory 310 and a processor 320. The memory 310 stores a computer program executed by the processor 320. When the computer program is executed by the processor 320, the processor 320 executes the speech enhancement method 100 or 200 according to the embodiment of the present application described above. Those skilled in the art can understand the specific operation of the speech enhancement device 300 according to the embodiment of the present application in combination with the above content. For the sake of brevity, the specific details are not repeated here.

[0056] Combine the following Figure 4 The speech enhancement system 400 provided according to another aspect of the present application is described. Figure 4 As shown, the voice enhancement system 400 may include an image acquisition device 410, a sound pickup device 420, and a voice enhancement device 430, wherein the image acquisition device 410 is used to acquire surveillance video of the reception site, the sound pickup device 420 is used to acquire audio recordings of the reception staff receiving customers, and the voice enhancement device 430 may adopt the voice enhancement device 300 described above, and is used to reduce noise on the audio recording based on the surveillance video and the audio recording. Those skilled in the art may understand the specific operation of the voice enhancement system 400 according to the embodiment of the present application in combination with the above-mentioned contents, and for the sake of brevity, it will not be repeated here.

[0057] In addition, according to an embodiment of the present application, a storage medium is also provided, on which program instructions are stored, and when the program instructions are executed by a computer or a processor, the corresponding steps of the speech enhancement method of the embodiment of the present application are executed. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0058] Based on the above description, according to the speech enhancement method, device and system of the embodiments of the present application, the sound pickup device is placed on one of the two objects involved in the reception process, and the object is locked in combination with the surveillance video and then the other object is located, and finally the relative position between the two is obtained for noise reduction, which can achieve high-quality noise reduction effect.

[0059] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application to this. Those of ordinary skill in the art may make various changes and modifications therein without departing from the scope and spirit of the present application. All these changes and modifications are intended to be included within the scope of the present application as required by the appended claims.

[0060] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0061] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0062] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0063] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present application should not be interpreted as reflecting the following intention: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with features less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0064] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0065] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0066] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some modules according to the embodiments of the present application. The application can also be implemented as a device program (e.g., computer program and computer program product) for executing a part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0067] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application can be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words target, second, and third, etc. does not indicate any order. These words can be interpreted as names.

[0068] The above is only a specific implementation or description of a specific implementation of the present application, and the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A speech enhancement method, characterized in that: The method comprises: Acquire a recording audio of the process of a target receptionist receiving a target customer, wherein the recording audio is picked up by a sound pickup device worn on the target receptionist; Obtain surveillance video of the same period as the recorded audio, and identify the target receptionist and other personnel from the surveillance video; Determine the target customer from the other personnel according to the target reception personnel; Determine the position information of the target customer relative to the target receptionist during the time period, and determine the noise reduction parameter according to the position information; Performing noise reduction processing on the recorded audio according to the noise reduction parameters; The step of determining the noise reduction parameters according to the orientation information includes: determining the parameters of a beamforming algorithm according to the orientation information.

2. The method according to claim 1, characterized in that The method further includes: obtaining the location information of the target receptionist during the process of the target receptionist receiving the target customer; The method of determining the target reception staff and other personnel from the surveillance video includes: obtaining the location information of each person included in the surveillance video during the time period; comparing the location information of the target reception staff with the location information of each person during the time period to determine the target reception staff from among the various personnel, and personnel other than the target reception staff are other personnel.

3. The method according to claim 2, characterized in that The step of comparing the location information of the target reception staff with the location information of each staff in the time period to determine the target reception staff from the various staff includes: Acquiring a moving track of the target receptionist in the process of receiving the target customer based on the location information of the target receptionist; Acquire the movement trajectory of each person in the time period based on the position information of each person in the time period; The movement trajectory of the target reception staff in the process of receiving the target customer is matched with the movement trajectory of each staff in the time period, so as to determine the target reception staff from among the various staff.

4. The method according to claim 1, characterized in that: The recorded audio includes identification information of the target receptionist, and determining the target receptionist and other personnel from the surveillance video includes: A feature corresponding to the identification information is acquired from a pre-stored feature library, and the target receptionist is determined from the surveillance video based on the feature, and persons other than the target receptionist are other persons.

5. The method according to claim 1, characterized in that The step of determining the target customer from the other personnel according to the target reception personnel includes: The dynamic distance information between the target receptionist and the surrounding persons in the monitoring video is obtained, and the target customer is determined from the other persons according to the dynamic distance information.

6. The method according to claim 5, characterized in that The obtaining of dynamic distance information between the target receptionist and the surrounding persons in the surveillance video, and determining the target customer from the other persons according to the dynamic distance information, includes: Obtaining dynamic changes in the distance between the target receptionist and the surrounding persons of the target receptionist in the surveillance video; The person whose distance from the target receptionist changes the least dynamically is determined as the target customer.

7. The method according to claim 1, characterized in that The determining of the position information of the target customer relative to the target receptionist during the time period is also based on the facial orientation information of the target receptionist during the time period.

8. The method according to claim 1, characterized in that The position information within the time period varies with time, and the parameters of the beamforming algorithm also vary with time.

9. The method according to claim 1, characterized in that: The relative position between the sound pickup device and the target receptionist is fixed, and the method further includes: Acquiring the position information of the sound pickup device relative to the target receptionist; The recorded audio is subjected to noise reduction processing based on the position information of the sound pickup device relative to the target receptionist.

10. The method according to claim 2, characterized in that The sound pickup device has a positioning function to obtain the position information of the target receptionist.

11. The method according to claim 2, characterized in that The obtaining of the location information of each person included in the surveillance video during the time period includes: Constructing a three-dimensional personnel model of each person included in the surveillance video according to the positions, angles and captured image ranges of multiple image acquisition devices that acquire the surveillance video; Each of the three-dimensional personnel models is identified and the position is tracked to obtain the position information of each person in the time period.

12. The method according to claim 11, characterized in that The recorded audio and the monitored video are both uploaded to the backend server in real time so that the voice enhancement method can be executed by the backend server.

13. The method according to any one of claims 1 to 12, characterized in that: The recorded audio after noise reduction processing is transcribed and semantically extracted to obtain the reception quality inspection result of the target reception staff, so as to realize the reception quality inspection of the target reception staff.

14. A speech enhancement method, characterized in that: The method comprises: Acquire a recording audio of the process in which the target receptionist receives the target customer, wherein the recording audio is picked up by a sound pickup device worn on the target customer; Obtaining surveillance video of the same period as the recorded audio, and identifying the target customer and other personnel from the surveillance video; Determine the target receptionist from the other personnel according to the target customer; determine the position information of the target receptionist relative to the target customer during the time period, and determine the noise reduction parameter according to the position information; Performing noise reduction processing on the recorded audio according to the noise reduction parameters; The step of determining the noise reduction parameters according to the orientation information includes: determining the parameters of a beamforming algorithm according to the orientation information.

15. A speech enhancement device, characterized in that: The device includes a memory and a processor, wherein a computer program executed by the processor is stored in the memory, and when the computer program is executed by the processor, the processor executes the speech enhancement method according to any one of claims 1 to 14.

16. A speech enhancement system, characterized in that: The system includes an image acquisition device, a sound pickup device and the voice enhancement device described in claim 15, wherein the image acquisition device is used to acquire surveillance video of the reception venue, the sound pickup device is used to acquire recorded audio of the process of reception staff receiving customers, and the voice enhancement device is used to reduce noise on the recorded audio based on the surveillance video and the recorded audio.

Citation Information

Patent Citations

  • Passenger flow volume statistical method and device based on voice recognition, equipment and medium

    CN110930990A

  • Sound processing method, device and equipment

    CN110970057A