Intelligent customer service man-machine interaction method and system and storage medium

Abstract

The invention provides an intelligent customer service man-machine interaction method and system and a storage medium. The method comprises the following steps: performing face detection on a camera video stream; if the face of the user is detected in the camera video stream, acquiring a video acquisition address of the camera video stream, and determining target pickup equipment according to the video acquisition address; controlling the target pickup device to perform voice acquisition to obtain acquired voice, and performing voice recognition on the acquired voice to obtain a voice text; interaction feedback information is determined according to the voice text, and the intelligent customer service is controlled to execute man-machine interaction according to the interaction feedback information. According to the embodiment of the invention, the target pickup device is controlled to carry out voice acquisition, voice acquisition can be effectively carried out on the user at the corresponding height, the acquired voice is obtained, and the intelligent customer service can be effectively controlled to execute man-machine interaction based on the interaction feedback information. Therefore, man-machine interaction service can be effectively provided for special groups with limited height, such as children, wheelchair disabled people, dwarf and the like.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to an intelligent customer service human-computer interaction method, system and storage medium. Background Art

[0002] Intelligent customer service human-computer interaction refers to the use of artificial intelligence technologies such as speech recognition, natural language processing, and image recognition to enable computers to understand human language, behavior, or instructions, and make corresponding feedback or operations. With the continuous advancement of technology, intelligent customer service human-computer interaction has become an important means for companies to improve service quality and optimize customer experience.

[0003] In the existing intelligent customer service human-computer interaction process, due to the limitation of the camera installation position, it is impossible to effectively provide human-computer interaction services for special groups with limited height, such as children, wheelchair-bound people, dwarfs, etc. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide an intelligent customer service human-computer interaction method, system and storage medium to solve the problem that the prior art cannot effectively provide effective human-computer interaction services for special groups with limited height.

[0005] The embodiment of the present invention is implemented as follows: an intelligent customer service human-computer interaction method, the method comprising:

[0006] Obtaining a camera video stream and performing face detection on the camera video stream;

[0007] If a user's face is detected in the camera video stream, a video acquisition address of the camera video stream is obtained, and a target sound pickup device is determined according to the video acquisition address;

[0008] Controlling the target sound pickup device to collect voice to obtain collected voice, and performing voice recognition on the collected voice to obtain voice text;

[0009] Determine interactive feedback information according to the voice text, and control the intelligent customer service to perform human-computer interaction according to the interactive feedback information.

[0010] Preferably, obtaining the video acquisition address of the camera video stream and determining the target sound pickup device according to the video acquisition address includes:

[0011] Obtaining a signal transmission address of the camera video stream, and determining the video acquisition address by using the signal transmission address;

[0012] Determine the camera position according to the video acquisition address, and calculate the distance between the camera position and each sound pickup device to obtain a first position distance;

[0013] The sound pickup device corresponding to the minimum first position distance is determined as the target sound pickup device.

[0014] Preferably, after performing face detection on the camera video stream, the method further comprises:

[0015] If the number of videos of the camera video stream in which the user's face is detected is greater than a preset number, setting the camera video stream as a candidate video stream;

[0016] Obtaining the camera positions corresponding to the candidate video streams respectively, and calculating the distance between each camera position and the default position respectively, to obtain a second position distance;

[0017] The candidate video stream corresponding to the minimum second position distance is set as the target video stream, and the target sound pickup device is determined according to the video acquisition address of the target video stream.

[0018] Preferably, performing speech recognition on the collected speech to obtain speech text includes:

[0019] Performing speaker detection on the collected speech, and judging whether there are at least two speakers in the collected speech according to the speaker detection result;

[0020] If there are at least two speakers in the collected voice, perform speaker segmentation on the collected voice to obtain speaker voices, and determine a target voice in the speaker voice according to the user face;

[0021] The target speech is converted into text to obtain the speech text.

[0022] Preferably, performing speaker detection on the collected speech, and judging whether there are at least two speakers in the collected speech according to the speaker detection result, comprises:

[0023] Performing zero-crossing detection on the collected speech, and performing speech segmentation on the collected speech according to the zero-crossing detection result to obtain segmented speech;

[0024] Acquire speech features of the segmented speech, and calculate similarities between different segmented speech according to the speech features to obtain a first feature similarity, and obtain the speaker detection result;

[0025] If the first feature similarity in the speaker detection result is less than the similarity threshold, determining the segmented speech corresponding to the first feature similarity as abnormal speech;

[0026] If the number of the abnormal voices is greater than the number threshold, it is determined that there are at least two speakers in the collected voice.

[0027] Preferably, performing speaker segmentation on the collected speech to obtain speaker speech includes:

[0028] The speaker audio in the collected voice is determined according to the voice feature, and the speaker audio corresponding to the voice feature is combined to obtain the speaker voice.

[0029] Preferably, determining the target voice in the speaker's voice according to the user's face includes:

[0030] If only one face of the user is detected in the camera video stream, the user face is determined as a target face;

[0031] If at least two user faces are detected in the camera video stream, the image areas of the respective user faces are calculated, and the user face corresponding to the largest image area is determined as the target face;

[0032] Determining a user's age and a user's gender according to the target face, and determining a voice reference feature according to the user's age and the user's gender;

[0033] The speech feature corresponding to the speaker's speech is calculated similarity with the speech reference feature to obtain a second feature similarity, and the speaker's speech corresponding to the maximum second feature similarity is determined as the target speech.

[0034] Another object of an embodiment of the present invention is to provide an intelligent customer service human-computer interaction system, the system comprising:

[0035] A face detection module is used to obtain a camera video stream and perform face detection on the camera video stream;

[0036] A device determination module, configured to obtain a video acquisition address of the camera video stream if a user's face is detected in the camera video stream, and determine a target sound pickup device according to the video acquisition address;

[0037] A speech recognition module, used for controlling the target sound pickup device to collect speech, obtain collected speech, and perform speech recognition on the collected speech to obtain speech text;

[0038] The human-computer interaction module is used to determine the interaction feedback information according to the voice text, and control the intelligent customer service to perform human-computer interaction according to the interaction feedback information.

[0039] Preferably, the device determination module is further used for:

[0040] Obtaining a signal transmission address of the camera video stream, and determining the video acquisition address by using the signal transmission address;

[0041] Determine the camera position according to the video acquisition address, and calculate the distance between the camera position and each sound pickup device to obtain a first position distance;

[0042] The sound pickup device corresponding to the minimum first position distance is determined as the target sound pickup device.

[0043] The embodiments of the present invention can effectively determine the target sound pickup device based on the video acquisition address by acquiring the video acquisition address of the camera video stream, and can effectively collect the voice of users of corresponding heights by controlling the target sound pickup device to obtain the collected voice. Interaction feedback information can be effectively determined based on the voice text corresponding to the collected voice, and intelligent customer service can be effectively controlled to perform human-computer interaction based on the interaction feedback information, thereby effectively providing human-computer interaction services for special groups with limited heights, such as children, wheelchair-bound people, and dwarfs. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the intelligent customer service human-computer interaction method provided by the first embodiment of the present invention;

[0045] Figure 2 It is a schematic diagram of specific implementation steps of the intelligent customer service human-computer interaction method provided by the first embodiment of the present invention;

[0046] Figure 3 It is a structural diagram of an intelligent customer service human-computer interaction system provided by a second embodiment of the present invention;

[0047] Figure 4 It is a schematic diagram of the structure of a terminal device provided in the third embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] In order to illustrate the technical solution of the present invention, a specific embodiment is provided below for illustration.

[0050] Embodiment 1

[0051] See also Figure 1 , is a flow chart of an intelligent customer service human-computer interaction method provided by a first embodiment of the present invention. The intelligent customer service human-computer interaction method can be applied to any device or system. The intelligent customer service human-computer interaction method comprises the steps of:

[0052] Step S10, obtaining a camera video stream, and performing face detection on the camera video stream;

[0053] In this embodiment, at least two cameras are provided, and the positions of the cameras can be set according to the requirements. The cameras are used to collect video of the corresponding area to obtain a camera video stream. In this step, face detection is performed on the camera video stream to determine whether there is a face in the camera video stream. If there is a face in the camera video stream, it is determined that a user currently needs to perform human-computer interaction. Preferably, face detection can be performed on the camera video stream based on a deep learning algorithm.

[0054] Step S20, if a user's face is detected in the camera video stream, obtaining a video acquisition address of the camera video stream, and determining a target sound pickup device according to the video acquisition address;

[0055] Among them, by obtaining the video acquisition address of the camera video stream, the target sound pickup device can be effectively determined based on the video acquisition address. The installation positions of different target sound pickup devices are different. In this embodiment, at least two target sound pickup devices are provided, and different target sound pickup devices are used to collect voices of users at different heights.

[0056] Optionally, obtaining a video acquisition address of the camera video stream, and determining a target sound pickup device according to the video acquisition address, includes:

[0057] Obtaining the signal transmission address of the camera video stream, and determining the video acquisition address by using the signal transmission address; wherein the signal transmission address is matched with an address query table to obtain a video acquisition address, and the address query table stores a correspondence between different signal transmission addresses and corresponding video acquisition addresses;

[0058] The camera position is determined according to the video acquisition address, and the distance between the camera position and each sound pickup device is calculated to obtain a first position distance; wherein the video acquisition address and the camera position are stored in a one-to-one correspondence, and by calculating the distance between the camera position and each sound pickup device, the sound pickup device closest to the camera position can be effectively determined;

[0059] The sound pickup device corresponding to the minimum distance from the first position is determined as the target sound pickup device; wherein the sound pickup device closest to the camera position is determined as the target sound pickup device. In this step, if the camera located at a low position captures a face, the sound pickup device closest to the camera located at a low position is determined as the target sound pickup device, thereby facilitating voice collection for special groups with limited height.

[0060] Furthermore, after performing face detection on the camera video stream, the method further includes:

[0061] If the number of videos of the camera video stream in which the user's face is detected is greater than a preset number, the camera video stream is set as a candidate video stream; wherein the preset number can be set according to demand, for example, the preset number can be set to 1, and when the number of videos of the camera video stream in which the user's face is detected is 2, both camera video streams are set as candidate video streams;

[0062] Obtaining the camera positions corresponding to the candidate video streams respectively, and calculating the distances between the camera positions and the default position respectively, to obtain the second position distance; wherein the default position can be set according to the requirements;

[0063] The candidate video stream corresponding to the minimum second position distance is set as the target video stream, and the target sound pickup device is determined according to the video acquisition address of the target video stream; wherein, by setting the candidate video stream corresponding to the minimum second position distance as the target video stream, and determining the target sound pickup device according to the video acquisition address of the target video stream, the sound pickup device corresponding to the camera closest to the default position can be effectively determined as the target sound pickup device.

[0064] Step S30, controlling the target sound pickup device to collect voice to obtain collected voice, and performing voice recognition on the collected voice to obtain voice text;

[0065] Among them, by controlling the target sound pickup device to collect voice, the voice of users at the corresponding height can be effectively collected to obtain the collected voice, and the interactive feedback information can be effectively determined based on the voice text corresponding to the collected voice.

[0066] Optionally, performing speech recognition on the collected speech to obtain speech text includes:

[0067] Performing speaker detection on the collected speech, and judging whether there are at least two speakers in the collected speech according to the speaker detection result; wherein, by performing speaker detection on the collected speech, it is possible to effectively judge whether there are multiple speakers in the collected speech, thereby improving the accuracy of subsequent human-computer interaction;

[0068] If there are at least two speakers in the collected voice, the collected voice is subjected to speaker segmentation to obtain speaker voices, and the target voice in the speaker voice is determined according to the user face; wherein, when there are multiple speakers in the collected voice, the speaker voice corresponding to each speaker can be effectively segmented by performing speaker segmentation on the collected voice, and the target voice in the speaker voice can be determined by the user face, and the voice instructions that need to be executed for human-computer interaction can be effectively determined to improve the accuracy of human-computer interaction.

[0069] The target speech is converted into text to obtain the speech text.

[0070] Further, performing speaker detection on the collected speech, and judging whether there are at least two speakers in the collected speech according to the speaker detection result, comprises:

[0071] Performing zero-crossing detection on the collected speech, and performing speech segmentation on the collected speech according to the zero-crossing detection result to obtain segmented speech; wherein, by performing zero-crossing detection on the collected speech, speech pause positions in the collected speech are obtained, and speech segmentation is performed on the collected speech based on the speech pause positions to obtain segmented speech;

[0072] Acquire speech features of the segmented speech, and calculate similarities between different segmented speech according to the speech features to obtain first feature similarities, and obtain the speaker detection result; wherein the speech features may be Mel frequency cepstral coefficients and rhythm features, and by calculating the similarities between different segmented speech, detect whether the speakers of different segmented speech are the same;

[0073] If the first feature similarity in the speaker detection result is less than the similarity threshold, the segmented speech corresponding to the first feature similarity is determined as an abnormal speech; wherein the similarity threshold can be set according to demand, and if the first feature similarity is less than the similarity threshold, it is determined that the speakers of the two segmented speech corresponding to the first feature similarity are different, and therefore, the segmented speech corresponding to the first feature similarity is determined as an abnormal speech;

[0074] If the number of the abnormal voices is greater than the number threshold, it is determined that there are at least two speakers in the collected voice; wherein the number threshold can be set according to demand.

[0075] Furthermore, the collected speech is subjected to speaker segmentation to obtain the speaker's speech, including:

[0076] The speaker audio in the collected voice is determined according to the voice feature, and the speaker audio corresponding to the voice feature is combined to obtain the speaker voice; wherein, the audio information corresponding to the speaker is determined based on the voice feature to obtain the speaker audio, and by combining the speaker audio corresponding to the same voice feature, the speaker voice corresponding to the same speaker can be effectively combined, which facilitates the recognition of the voice command of the same speaker.

[0077] Preferably, determining the target voice in the speaker's voice according to the user's face includes:

[0078] If only one face of the user is detected in the camera video stream, the user face is determined as a target face;

[0079] If at least two user faces are detected in the camera video stream, the image areas of the user faces are calculated respectively, and the user face corresponding to the largest image area is determined as the target face; wherein, if at least two user faces are detected in the same camera video stream, it is necessary to determine the human-computer interaction priority between the users corresponding to the two user faces, therefore, by calculating the image areas of the user faces respectively, the human-computer interaction priority is determined based on the image areas, that is, the user face corresponding to the largest image area is determined as the target face with a higher human-computer interaction priority;

[0080] Determine the user age and user gender based on the target face, and determine the voice reference features based on the user age and user gender; wherein, by extracting key features from the target face, such as the shape of the eyes, the curvature of the eyebrows, the width of the nose, the thickness of the lips, etc., use principal component analysis (PCA), linear discriminant analysis (LDA) and other methods to convert the key features in the target face into feature representations, and determine the user gender based on the feature representations. Extract texture features from the target face, and determine the user gender based on the texture features, the texture features including information such as skin texture, number of wrinkles, and image color changes. In this step, match the determined user age and user gender with the feature query table to obtain the voice reference features, and the feature query table stores the correspondence between different user ages, user genders and corresponding voice reference features;

[0081] The similarity between the speech feature corresponding to the speaker's speech and the speech reference feature is calculated to obtain a second feature similarity, and the speaker's speech corresponding to the maximum second feature similarity is determined as the target speech; wherein, the Euclidean distance formula can be used to calculate the similarity between the speech feature and the speech reference feature to obtain the second feature similarity, and based on the maximum second feature similarity, the speaker's speech that best matches the target face with a higher human-computer interaction priority can be effectively determined as the target speech, thereby improving the accuracy of the target speech determination.

[0082] Step S40, determining interaction feedback information according to the voice text, and controlling the intelligent customer service to perform human-computer interaction according to the interaction feedback information;

[0083] The voice text is matched with the feedback information query table to obtain interactive feedback information, and the feedback information query table stores the corresponding relationship between different voice texts and corresponding interactive feedback information.

[0084] Optional, see Figure 2 In this embodiment, a dual microphone array (camera + microphone module) can also be used to obtain the picture information in front of the intelligent customer service interaction machine in real time through the cameras in the two arrays.

[0085] When the camera video stream in the upper array detects a face, the upper array microphone module is turned on to pick up the sound, while the lower array camera video stream is shielded. The face detection information and the voice recognized by the voice industrial computer are converted into text and given to the intelligent customer service interaction machine. The subsequent interaction process is carried out entirely using the upper microphone array video stream and audio stream.

[0086] When the face leaves, the upper microphone module is turned off to pick up sound, and the lower array camera video stream is turned on to obtain the video stream, and the intelligent customer service interaction machine is notified that the face has left;

[0087] When the camera video stream in the lower array detects a face and the camera in the upper array does not detect a face, the lower array microphone module is turned on to pick up the sound, and the upper array camera video stream is shielded at the same time. The face detection information and the voice recognized by the voice industrial computer are converted into text and given to the intelligent customer service interaction machine. The subsequent interaction process is carried out entirely using the lower microphone array video stream and audio stream.

[0088] When the face leaves, the lower microphone module is turned off to pick up the sound, and the upper array camera video stream is turned on to obtain the video stream, and the intelligent customer service interaction machine is notified that the face has left;

[0089] When the camera video streams in the upper and lower arrays detect a face at the same time, the upper microphone array is used as the main one by default, and the upper array microphone module is turned on to pick up sound, while the lower array camera video stream is shielded, and the face detection information and the voice recognized by the voice industrial computer are converted into text and given to the intelligent customer service interaction machine. The subsequent interaction process is carried out throughout the upper microphone array video stream and audio stream. When the face leaves, the upper microphone module is turned off to pick up sound, and the lower array camera video stream is turned on to obtain, and the intelligent customer service interaction machine is notified that the face has left.

[0090] Through the above mechanism, we can provide active intelligent interactive services to special groups with limited height, such as children, the disabled, dwarfs, etc., and expand the service audience of intelligent customer service interactive machines.

[0091] By utilizing the hardware configuration of a dual-microphone array and the software control of the voice industrial computer that automatically opens the corresponding microphone for sound pickup based on the face detection results and shields one video stream, the intelligent customer service interaction machine can basically meet the intelligent interaction needs of users within 30 to 50 centimeters and within a height range of 0.8 to 1.9 meters.

[0092] In this embodiment, by obtaining the video acquisition address of the camera video stream, the target sound pickup device can be effectively determined based on the video acquisition address, and by controlling the target sound pickup device to perform voice acquisition, the voice of users of corresponding height can be effectively acquired to obtain the acquired voice, and the interactive feedback information can be effectively determined based on the voice text corresponding to the acquired voice, and the intelligent customer service can be effectively controlled to perform human-computer interaction based on the interactive feedback information, thereby effectively providing human-computer interaction services for special groups with limited height, such as children, wheelchair-bound people, dwarfs, etc.

[0093] Embodiment 2

[0094] See also Figure 3 , is a schematic diagram of the structure of an intelligent customer service human-computer interaction system 100 provided in a second embodiment of the present invention, including:

[0095] The face detection module 10 is used to obtain the camera video stream and perform face detection on the camera video stream.

[0096] The device determination module 11 is used to obtain the video acquisition address of the camera video stream if a user face is detected in the camera video stream, and determine the target sound pickup device according to the video acquisition address.

[0097] Optionally, the device determination module 11 is further used to: obtain a signal transmission address of the camera video stream, and determine the video acquisition address by using the signal transmission address;

[0098] Determine the camera position according to the video acquisition address, and calculate the distance between the camera position and each sound pickup device to obtain a first position distance;

[0099] The sound pickup device corresponding to the minimum first position distance is determined as the target sound pickup device.

[0100] Further, the device determination module 11 is also used to: if the number of videos of the camera video stream in which the user's face is detected is greater than a preset number, set the camera video stream as a candidate video stream;

[0101] Obtaining the camera positions corresponding to the candidate video streams respectively, and calculating the distance between each camera position and the default position respectively, to obtain a second position distance;

[0102] The candidate video stream corresponding to the minimum second position distance is set as the target video stream, and the target sound pickup device is determined according to the video acquisition address of the target video stream.

[0103] The speech recognition module 12 is used to control the target sound pickup device to collect speech to obtain collected speech, and perform speech recognition on the collected speech to obtain speech text.

[0104] Optionally, the speech recognition module 12 is further used to: perform speaker detection on the collected speech, and determine whether there are at least two speakers in the collected speech according to the speaker detection result;

[0105] If there are at least two speakers in the collected voice, perform speaker segmentation on the collected voice to obtain speaker voices, and determine a target voice in the speaker voice according to the user face;

[0106] The target speech is converted into text to obtain the speech text.

[0107] Furthermore, the speech recognition module 12 is further used to: perform zero-crossing detection on the collected speech, and perform speech segmentation on the collected speech according to the zero-crossing detection result to obtain segmented speech;

[0108] Acquire speech features of the segmented speech, and calculate similarities between different segmented speech according to the speech features to obtain a first feature similarity, and obtain the speaker detection result;

[0109] If the first feature similarity in the speaker detection result is less than the similarity threshold, determining the segmented speech corresponding to the first feature similarity as abnormal speech;

[0110] If the number of the abnormal voices is greater than the number threshold, it is determined that there are at least two speakers in the collected voice.

[0111] Preferably, the speech recognition module 12 is further used to: determine the speaker audio in the collected speech according to the speech feature, and perform audio combination on the speaker audio corresponding to the speech feature to obtain the speaker speech.

[0112] Optionally, the speech recognition module 12 is further used to: if only one face of the user is detected in the camera video stream, determine the face of the user as the target face;

[0113] If at least two user faces are detected in the camera video stream, the image areas of the respective user faces are calculated, and the user face corresponding to the largest image area is determined as the target face;

[0114] Determining a user's age and a user's gender according to the target face, and determining a voice reference feature according to the user's age and the user's gender;

[0115] The speech feature corresponding to the speaker's speech is calculated similarity with the speech reference feature to obtain a second feature similarity, and the speaker's speech corresponding to the maximum second feature similarity is determined as the target speech.

[0116] The human-computer interaction module 13 is used to determine the interaction feedback information according to the voice text, and control the intelligent customer service to perform human-computer interaction according to the interaction feedback information.

[0117] In this embodiment, by obtaining the video acquisition address of the camera video stream, the target sound pickup device can be effectively determined based on the video acquisition address, and by controlling the target sound pickup device to perform voice acquisition, the voice of users of corresponding height can be effectively acquired to obtain the acquired voice, and the interactive feedback information can be effectively determined based on the voice text corresponding to the acquired voice, and the intelligent customer service can be effectively controlled to perform human-computer interaction based on the interactive feedback information, thereby effectively providing human-computer interaction services for special groups with limited height, such as children, wheelchair-bound people, dwarfs, etc.

[0118] Embodiment 3

[0119] Figure 4 2 is a block diagram of a terminal device 2 provided in the third embodiment of the present application. Figure 4 As shown, the terminal device 2 of this embodiment includes: a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program of the intelligent customer service human-computer interaction method. When the processor 20 executes the computer program 22, the steps in each embodiment of the above-mentioned intelligent customer service human-computer interaction method are implemented.

[0120] Exemplarily, the computer program 22 may be divided into one or more modules, which are stored in the memory 21 and executed by the processor 20 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 22 in the terminal device 2. The terminal device may include, but is not limited to, a processor 20 and a memory 21.

[0121] The processor 20 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0122] The memory 21 may be an internal storage unit of the terminal device 2, such as a hard disk or memory of the terminal device 2. The memory 21 may also be an external storage device of the terminal device 2, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 2. Further, the memory 21 may also include both an internal storage unit and an external storage device of the terminal device 2. The memory 21 is used to store the computer program and other programs and data required by the terminal device. The memory 21 may also be used to temporarily store data that has been output or is to be output.

[0123] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional unit.

[0124] If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Among them, the computer-readable storage medium can be non-volatile or volatile. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in computer-readable storage media can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable storage media do not include electrical carrier signals and telecommunication signals.

[0125] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An intelligent customer service human-computer interaction method, characterized in that: The method comprises: Obtaining a camera video stream and performing face detection on the camera video stream; If a user's face is detected in the camera video stream, a video acquisition address of the camera video stream is obtained, and a target sound pickup device is determined according to the video acquisition address; Controlling the target sound pickup device to collect voice to obtain collected voice, and performing voice recognition on the collected voice to obtain voice text; Determine interactive feedback information according to the voice text, and control the intelligent customer service to perform human-computer interaction according to the interactive feedback information.

2. The intelligent customer service human-computer interaction method according to claim 1, characterized in that: Obtaining the video acquisition address of the camera video stream, and determining the target sound pickup device according to the video acquisition address, including: Obtaining a signal transmission address of the camera video stream, and determining the video acquisition address by using the signal transmission address; Determine the camera position according to the video acquisition address, and calculate the distance between the camera position and each sound pickup device to obtain a first position distance; The sound pickup device corresponding to the minimum first position distance is determined as the target sound pickup device.

3. The intelligent customer service human-computer interaction method according to claim 2, characterized in that: After performing face detection on the camera video stream, the method further includes: If the number of videos of the camera video stream in which the user's face is detected is greater than a preset number, setting the camera video stream as a candidate video stream; Obtaining the camera positions corresponding to the candidate video streams respectively, and calculating the distance between each camera position and the default position respectively, to obtain a second position distance; The candidate video stream corresponding to the minimum second position distance is set as the target video stream, and the target sound pickup device is determined according to the video acquisition address of the target video stream.

4. The intelligent customer service human-computer interaction method according to claim 1, characterized in that: Performing speech recognition on the collected speech to obtain speech text includes: Performing speaker detection on the collected speech, and judging whether there are at least two speakers in the collected speech according to the speaker detection result; If there are at least two speakers in the collected voice, perform speaker segmentation on the collected voice to obtain speaker voices, and determine a target voice in the speaker voice according to the user face; The target speech is converted into text to obtain the speech text.

5. The intelligent customer service human-computer interaction method according to claim 4, characterized in that: Performing speaker detection on the collected speech, and judging whether there are at least two speakers in the collected speech according to the speaker detection result, comprises: Performing zero-crossing detection on the collected speech, and performing speech segmentation on the collected speech according to the zero-crossing detection result to obtain segmented speech; Acquire speech features of the segmented speech, and calculate similarities between different segmented speech according to the speech features to obtain a first feature similarity, and obtain the speaker detection result; If the first feature similarity in the speaker detection result is less than the similarity threshold, determining the segmented speech corresponding to the first feature similarity as abnormal speech; If the number of the abnormal voices is greater than the number threshold, it is determined that there are at least two speakers in the collected voice.

6. The intelligent customer service human-computer interaction method according to claim 4, characterized in that: Segmenting the collected speech by speaker to obtain the speaker's speech includes: The speaker audio in the collected voice is determined according to the voice feature, and the speaker audio corresponding to the voice feature is combined to obtain the speaker voice.

7. The intelligent customer service human-computer interaction method according to claim 5, characterized in that: Determining a target voice in the speaker's voice according to the user's face includes: If only one face of the user is detected in the camera video stream, the user face is determined as a target face; If at least two user faces are detected in the camera video stream, the image areas of the respective user faces are calculated, and the user face corresponding to the largest image area is determined as the target face; Determining a user's age and a user's gender according to the target face, and determining a voice reference feature according to the user's age and the user's gender; The speech feature corresponding to the speaker's speech is calculated similarity with the speech reference feature to obtain a second feature similarity, and the speaker's speech corresponding to the maximum second feature similarity is determined as the target speech.

8. An intelligent customer service human-computer interaction system, characterized in that: The system comprises: A face detection module is used to obtain a camera video stream and perform face detection on the camera video stream; A device determination module, configured to obtain a video acquisition address of the camera video stream if a user's face is detected in the camera video stream, and determine a target sound pickup device according to the video acquisition address; A speech recognition module, used for controlling the target sound pickup device to collect speech, obtain collected speech, and perform speech recognition on the collected speech to obtain speech text; The human-computer interaction module is used to determine the interaction feedback information according to the voice text, and control the intelligent customer service to perform human-computer interaction according to the interaction feedback information.

9. The intelligent customer service human-computer interaction system according to claim 8, characterized in that: The device determination module is also used for: Obtaining a signal transmission address of the camera video stream, and determining the video acquisition address by using the signal transmission address; Determine the camera position according to the video acquisition address, and calculate the distance between the camera position and each sound pickup device to obtain a first position distance; The sound pickup device corresponding to the minimum first position distance is determined as the target sound pickup device.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images