Scene perception and interaction method and system for intelligent video conference, medium and product

Abstract

The invention discloses a scene perception and interaction method and system for an intelligent video conference, a medium and a product, and relates to the technical field of intelligent conferences, and the method specifically comprises the steps: carrying out the photographing of a conference site through an image detection module, so as to obtain first image data containing a plurality of participants, screening the plurality of participants according to the first image data to determine a target speaking person; obtaining each target sound signal contained in the conference site through an audio detection module, and determining a speaker verification result corresponding to the target speaker according to each target sound signal; when it is detected that the speaker verification result is that verification is passed, second image data containing the target speaker is shot through the image detection module; and generating a first conference video based on the second image data and each target sound signal, and sending the first conference video to a preset target conference interface. According to the invention, the accuracy of the judgment result of the intelligent conference system on the current spokesman can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent conference technology, and in particular to a scene perception and interaction method for intelligent video conferencing, an intelligent conference system, a storage medium, and a computer program product. Background Art

[0002] With the continuous development of smart conference technology, smart conferences have become an indispensable and efficient tool in business negotiations, academic exchanges and other activities.

[0003] In related technologies, intelligent conference systems usually call on a camera device pre-installed at the conference site to identify the face of each participant, thereby capturing the facial features of each participant, and judging the current speaker based on each facial feature to dynamically generate a conference screen, thereby ensuring that each participant can understand the speech information of the current speaker through the conference screen.

[0004] However, when the lighting conditions at the conference site change significantly, the facial images captured by the camera device usually have defects such as facial blur and loss of details, which leads to reduced accuracy of the intelligent conference system's judgment results on the current speaker, thereby greatly reducing the efficiency of the meeting. Summary of the invention

[0005] The main purpose of this application is to provide a scene perception and interaction method for intelligent video conferencing, an intelligent conferencing system, a storage medium and a computer program product, aiming to solve the technical problem in related technologies that the intelligent conferencing system has low accuracy in judging the current speaker.

[0006] To achieve the above objectives, the present application proposes a scene perception and interaction method for intelligent video conferencing, which is applied to an intelligent conferencing system. The intelligent conferencing system includes an image detection module and an audio detection module. The scene perception and interaction method for intelligent video conferencing includes: The conference site is photographed by the image detection module to capture first image data containing a plurality of conference participants, and the plurality of conference participants are screened according to the first image data to determine a target speaker; Acquire each target sound signal contained in the conference site through the audio detection module, and determine the speaker verification result corresponding to the target speaker according to each target sound signal; When it is detected that the speaker verification result is passed, capturing second image data containing the target speaker through the image detection module, wherein the target speaker is located at the center of the second image data; A first conference video is generated based on the second image data and each of the target sound signals, and the first conference video is sent to a preset target conference interface so as to display the first conference video to other online participants through the target conference interface.

[0007] In one embodiment, the step of screening the plurality of conference participants according to the first image data to determine a target speaker includes: extracting image features contained in the first image data; Determine facial features of the multiple conference participants that match each other based on the image features, and determine facial movement features included in the facial features of each of the participants; The conference participant corresponding to the facial movement feature is determined as the target speaker.

[0008] In one embodiment, the audio detection module includes a first detection unit and a second detection unit, and the step of acquiring each target sound signal contained in the conference site through the audio detection module includes: The first detection unit collects a first initial sound signal contained in the conference site, and the second detection unit collects a second initial sound signal contained in the conference site; The first initial sound signal and the second initial sound signal are subjected to noise reduction processing to obtain a first target sound signal and a second target sound signal.

[0009] In one embodiment, the step of determining the speaker verification result corresponding to the target speaker according to each of the target sound signals includes: Determining a first signal reception time corresponding to the first target sound signal and a second signal reception time corresponding to the second target sound signal; Determine the sound source location information according to the first signal receiving time and the second signal receiving time; Determine the speaker position information of the target speaker based on the first image data, and compare the sound source position information with the speaker position information to obtain a comparison result; In the case where it is detected that the comparison result is that the sound source position information matches the speaker position information, determining that the speaker verification result is verification passed; When it is detected that the comparison result is that the sound source position information and the speaker position information do not match, it is determined that the speaker verification result is a verification failure.

[0010] In one embodiment, after the step of sending the first conference video to a preset target conference interface, the method further includes: Capturing a plurality of third image data including the target speaker through the image detection module; Determine the moving trajectory of the target speaker based on the plurality of third image data, and determine the initial shooting angle of the image detection module; generating an angle adjustment parameter based on the movement trajectory and the initial shooting angle, and adjusting the initial shooting angle to a target shooting angle according to the angle adjustment parameter; Controlling the image detection module to capture fourth image data including the target speaker according to the target shooting angle; A second conference video is generated based on the fourth image data and each of the target sound signals, and the second conference video is sent to the target conference interface so as to display the second conference video to the other online participants through the target conference interface.

[0011] In one embodiment, after the step of sending the first conference video to a preset target conference interface, the method further includes: Determining the identity information of the target speaker; Based on the identity information, the first conference video is updated to obtain a third conference video; The third conference video is sent to the target conference interface so as to display the third conference video to the other online participants through the target conference interface.

[0012] In one embodiment, after the step of sending the first conference video to a preset target conference interface, the method further includes: Receiving an interactive voice signal triggered by the target speaker, and determining a target interactive instruction according to the interactive voice signal; In the case where it is detected that the instruction type of the target interaction instruction is an environment control instruction, determining a target environment adjustment device corresponding to the conference site based on the environment control instruction; The target environment adjustment device is controlled according to the environment control instruction, so as to adjust the environmental parameters of the conference site through the target environment adjustment device.

[0013] In addition, to achieve the above-mentioned objectives, the present application also proposes an intelligent conference system, which includes: an image detection module, an audio detection module, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the scene perception and interaction method of the intelligent video conferencing as described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the scene perception and interaction method of the intelligent video conferencing as described above are implemented.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the scene perception and interaction method of the intelligent video conferencing as described above.

[0016] An embodiment of the present application provides a scene perception and interaction method for intelligent video conferencing, which is applied to an intelligent conference system. The intelligent conference system includes an image detection module and an audio detection module. The image detection module is used to shoot the conference scene to capture first image data containing multiple participants, and the multiple participants are screened according to the first image data to determine a target speaker; the audio detection module is used to obtain each target sound signal contained in the conference scene, and a speaker verification result corresponding to the target speaker is determined according to each target sound signal; when it is detected that the speaker verification result is a passed verification, the image detection module is used to capture second image data containing the target speaker, wherein the target speaker is at the center of the second image data; a first conference video is generated based on the second image data and each target sound signal, and the first conference video is sent to a preset target conference interface, so as to display the first conference video to other online participants through the target conference interface.

[0017] In this embodiment, during the operation of the intelligent conference system, the image detection module configured at the conference site is first called to shoot multiple participants participating in the meeting to capture first image data containing multiple participants. The intelligent conference system then identifies the first image data to determine the target speaker who is speaking among the multiple participants. After that, the intelligent conference system calls the audio detection module configured at the conference site to collect each target sound signal emitted by the target speaker at the conference site. The intelligent conference system then processes the target sound signal to determine the speaker verification result based on the target sound signal. After that, when the intelligent conference system detects that the speaker verification result is passed, the image detection module is called to shoot the target speaker to capture second image data containing the target speaker and the target speaker is at the center of the image. Finally, the intelligent conference system fuses the second image data and each target sound signal to generate a first conference video, and sends the first conference video to the preset target conference interface so that other online participants can view the first conference video through the target conference interface.

[0018] In this way, the present application solves the technical problem of low accuracy of the judgment result of the current speaker by the intelligent conference system in the related technology, that is, the present application identifies multiple participants by calling the image detection module to determine the target speaker who is speaking, and determines whether the detected target speaker is accurate through the audio detection module, so that when it is determined that the detected target speaker is accurate, the image data containing the target speaker and the collected target sound signal are fused to generate a conference video, and the conference video is sent to the conference interface, so that the intelligent conference system can identify the location information of the target speaker who is speaking according to the sound signal during operation, and thus determine whether the identified speaker is accurate according to the location information, thereby enabling the intelligent conference system to accurately identify the target speaker at the conference site when the on-site environment changes significantly, thereby achieving the technical effect of improving the accuracy of the judgment result of the intelligent conference system on the current speaker, and greatly improving the efficiency of the meeting. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A flowchart of the first embodiment of the scene perception and interaction method for intelligent video conferencing provided in the present application; Figure 2 A schematic diagram of a conference site involved in an embodiment of a scene perception and interaction method for intelligent video conferencing of the present application; Figure 3 A flow chart of the second embodiment of the scene perception and interaction method for intelligent video conferencing provided in this application; Figure 4 A brief flow chart of the scene perception and interaction method of the intelligent video conference for this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the scene perception and interaction method of intelligent video conferencing in the embodiment of the present application.

[0022] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0023] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0024] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0025] In this embodiment, for ease of description, the following is an intelligent conference system equipped with an image detection module, an audio detection module, and a data processing module, or a mobile terminal, a data storage control terminal, a PC, and other terminals connected to an electronic control unit supporting the intelligent conference system as the execution subject. Among them, the image detection module is composed of one or more camera devices configured at the conference site, and similarly, the audio detection module is composed of multiple audio detection units deployed at different locations at the conference site.

[0026] Based on the above-mentioned intelligent conference system, the overall concept of the scene perception and interaction method of the intelligent video conferencing of this application is proposed here.

[0027] With the continuous development of intelligent conference technology, intelligent conferences have become an indispensable and efficient tool in business negotiations, academic exchanges and other activities. In related technologies, intelligent conference systems usually call the camera device pre-set at the conference site to identify the faces of each participant, thereby capturing the facial features of each participant, and judging the current speaker based on each facial feature to dynamically generate a conference screen, thereby ensuring that each participant can understand the speech information of the current speaker through the conference screen. However, when the lighting conditions at the conference site change significantly, the facial images obtained by the camera device usually have defects such as facial blur and loss of details, which leads to a decrease in the accuracy of the intelligent conference system's judgment result on the current speaker, thereby greatly reducing the efficiency of the meeting.

[0028] In view of the above phenomenon, the present application provides a scene perception and interaction method for intelligent video conferencing, which is applied to an intelligent conference system. The intelligent conference system includes an image detection module and an audio detection module. The scene perception and interaction method for intelligent video conferencing includes: shooting the conference site through the image detection module to capture first image data containing multiple participants, and screening the multiple participants according to the first image data to determine the target speaker; acquiring each target sound signal contained in the conference site through the audio detection module, and determining the speaker verification result corresponding to the target speaker according to each target sound signal; when it is detected that the speaker verification result is verified, capturing second image data containing the target speaker through the image detection module, wherein the target speaker is at the center of the second image data; generating a first conference video based on the second image data and each target sound signal, and sending the first conference video to a preset target conference interface, so as to display the first conference video to other online participants through the target conference interface.

[0029] In this way, the present application solves the technical problem of low accuracy of the judgment result of the current speaker by the intelligent conference system in the related technology, that is, the present application identifies multiple participants by calling the image detection module to determine the target speaker who is speaking, and determines whether the detected target speaker is accurate through the audio detection module, so that when it is determined that the detected target speaker is accurate, the image data containing the target speaker and the collected target sound signal are fused to generate a conference video, and the conference video is sent to the conference interface, so that the intelligent conference system can identify the location information of the target speaker who is speaking according to the sound signal during operation, and thus determine whether the identified speaker is accurate according to the location information, thereby enabling the intelligent conference system to accurately identify the target speaker at the conference site when the on-site environment changes significantly, thereby achieving the technical effect of improving the accuracy of the judgment result of the intelligent conference system on the current speaker, and greatly improving the efficiency of the meeting.

[0030] Based on the overall concept of the scene perception and interaction method of the intelligent video conference of the present application, the embodiment of the present application provides a scene perception and interaction method of the intelligent video conference, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the scene perception and interaction method for intelligent video conferencing of the present application.

[0031] In this embodiment, the scene perception and interaction method of the intelligent video conference is applied to an intelligent conference system, and the intelligent conference system includes a wireless signal transmission module. The scene perception and interaction method of the intelligent video conference includes steps S10 to S40: Step S10: photographing the conference site through the image detection module to capture first image data containing multiple conference participants, and screening the multiple conference participants according to the first image data to determine the target speaker; It should be noted that the first image data is image data of multiple participants captured by the image detection module when shooting the conference site. It is understandable that the image detection module is composed of one or more camera devices set at the conference site, and each camera device should be set at a position where the face of the participant can be captured. In addition, the target speaker is the participant who is speaking among the multiple participants.

[0032] In this embodiment, during the operation of the intelligent conference system, the image detection module configured in itself is first controlled, so that the image detection module calls multiple cameras included in the scene to shoot the conference scene to obtain multiple image data to be fused. The image detection module then fuses the multiple image data to be fused to obtain the first image data containing the faces of all participants. The image detection module sends the first image data to the data processing module configured in the intelligent conference system, and the data processing module screens the multiple participants based on the first image data to determine the target speaker who is speaking.

[0033] For example, see Figure 2 , Figure 2 This is a schematic diagram of a conference scene involved in an embodiment of a scene perception and interaction method for intelligent video conferencing of the present application, such as Figure 2 As shown, during the operation of the intelligent conference system, the image detection module configured by itself is first controlled, so that the image detection module calls the camera A, camera B, and camera C configured on the scene to shoot the conference scene to capture multiple initial image data containing participants. The image detection module then splices the multiple initial image data obtained to eliminate the overlapping parts between the multiple initial image data, so as to obtain the first image data containing the faces of all participants. The image detection module then inputs the obtained first image data into the data processing module configured in the intelligent conference system, and the data processing module identifies the first image data to extract the facial features of each person contained in the first image data that match each participant. The data processing module screens each participant according to the facial features of each person to determine the target speaker who is speaking.

[0034] In addition, in this embodiment and another embodiment, in addition to calling multiple cameras to simultaneously shoot the scene, the image detection module can also directly call camera A, camera B, and camera C to shoot the meeting scene when it is detected that one of the cameras can capture the faces of all participants, thereby capturing first image data containing multiple participants.

[0035] In this way, during the meeting, the intelligent conference system can call the image detection module to shoot the meeting scene, thereby capturing the first image data containing all participants, and determine the target speaker who is speaking among all participants based on the first image data.

[0036] In a feasible implementation manner, the step of "screening the plurality of conference participants according to the first image data to determine the target speaker" in the above step S10 may specifically include steps S101 to S103: Step S101: extracting image features contained in the first image data; Step S102: determining facial features of each of the multiple conference participants that match the respective features based on the image features, and determining facial movement features included in the facial features of each of the participants; Step S103: Determine the participant corresponding to the facial movement feature as the target speaker.

[0037] In this embodiment, after receiving the first image data input by the image detection module, the data processing module first extracts the image features contained in the first image data. Then, based on the image features, the data processing module identifies the facial features corresponding to the faces of multiple participants. The data processing module then screens the facial features of each person to determine the facial movement features of the face that have changed. Finally, the data processing module determines the participant whose facial movement features are matched in the first image data, and determines the participant as the target speaker who is speaking.

[0038] Exemplarily, for example, after receiving the first image data input by the image detection module, the data processing module first extracts the image features contained in the first image data. Then, the data processing module processes the image features to obtain the facial features of each participant that match the first image data through image feature extraction. At this time, the data processing module reads the storage module configured in the intelligent conference system to obtain multiple preset standard movement features. The data processing module then matches the facial features of each person with the multiple standard movement features respectively, thereby determining the facial movement features that match the standard movement features among the multiple facial features of the people. Finally, the data processing module determines the participant whose facial movement features match the first image data as the target speaker who is speaking.

[0039] In this way, during the meeting, the intelligent conference system can call the image detection module to shoot the meeting scene, thereby capturing the first image data containing all participants, and determine the target speaker who is speaking among all participants based on the first image data.

[0040] Step S20: acquiring each target sound signal contained in the conference site through the audio detection module, and determining the speaker verification result corresponding to the target speaker according to each target sound signal; It should be noted that the target sound signal is a voice signal generated by a participant who is speaking at the conference site and is collected by the audio detection module. In addition, the speaker verification result is a verification result for determining whether the identified target speaker is accurate. The speaker verification result includes verification pass and verification fail. When the speaker verification result is verification pass, it is determined that the identified target speaker is correct. Similarly, when the speaker verification result is verification fail, it is determined that the identified target speaker is wrong.

[0041] In this embodiment, after determining that the data processing module has screened out the target speaker, the intelligent conference system further controls the audio detection module configured by itself, so that the audio detection module calls multiple audio detection units configured at the conference site to collect the target sound signals generated by the target speaker when speaking at the conference site. The audio detection module then inputs the target sound signals into the data processing unit, and the data processing unit determines whether the screened out target speaker is correct based on the target sound signals, and generates a corresponding speaker verification result.

[0042] For example, after the intelligent conference system determines that the data processing module has screened out the target speaker from among the multiple participants, it further controls the audio detection module configured by itself, and the audio detection module then calls the following Figure 2 As shown, the first microphone (i.e. Figure 2 The first detection unit in the Figure 2 The second detection unit in the audio detection module respectively collects the target sound signals generated by the target speaker during the speech. The audio detection module then sends the target sound signals collected by the first microphone and the second microphone to the above-mentioned data processing unit, and the data processing unit recognizes each acquired target sound signal to determine whether the target speaker recognized based on the above-mentioned first image data is correct according to the recognition result, and generates a corresponding speaker verification result.

[0043] In this way, after screening out the target speaker, the intelligent conference system can further collect the target sound signal generated by the target speaker during the speech through multiple audio collection units in the audio detection module, and identify the accuracy of the identified target speaker based on the target sound signal.

[0044] In a feasible implementation, the audio detection module includes a first detection unit and a second detection unit, and the step of "obtaining each target sound signal contained in the conference site through the audio detection module" in the above step S20 may specifically include steps S201-S202: Step S201: collecting a first initial sound signal contained in the conference site by the first detection unit, and collecting a second initial sound signal contained in the conference site by the second detection unit; Step S202: performing noise reduction processing on the first initial sound signal and the second initial sound signal to obtain a first target sound signal and a second target sound signal.

[0045] In this embodiment, after determining that the data processing module has screened out the target speaker, the intelligent conference system further controls the audio detection module configured by itself, so that the audio detection module calls the first detection unit configured on site to collect and detect the conference site, so as to collect the first initial sound signal generated by the target speaker when speaking at the conference site. At the same time, the audio detection module calls the second detection unit configured on site to collect and detect the conference site, so as to collect the second initial sound signal generated by the target speaker when speaking at the conference site. Afterwards, the audio detection module processes the acquired first initial sound signal and the second initial sound signal to extract the sound signal features contained in the first initial sound signal and the second initial sound signal respectively, and performs noise reduction processing according to the features of each sound signal to obtain the first target sound signal and the second target sound signal that do not contain noise signals.

[0046] Exemplarily, for example, after determining that the data processing module has screened out the target speaker, the intelligent conference system further controls the audio detection module configured by itself, and the audio detection module then calls the first microphone configured in the conference site to collect the voice signal generated by the target speaker during the speech to obtain a first initial sound signal. At the same time, the audio detection module then calls the second microphone configured at another location in the conference site to collect the voice signal generated by the target speaker during the speech to obtain a second initial sound signal. Thereafter, the audio detection module processes the acquired first initial sound signal and the second initial sound signal to extract the sound signal features corresponding to the first initial sound signal and the second initial sound signal respectively. At the same time, the audio detection module reads the above-mentioned storage module to obtain the preset standard noise features. The audio detection module then compares each sound signal feature with the standard noise feature to determine whether the sound signal feature and the noise signal feature match. When the audio detection module determines that the sound signal feature and the noise signal feature match, it determines that there is a noise signal in the first initial sound signal and the second initial sound signal. Finally, the audio detection module denoises the noise signal to obtain the first target sound signal and the second target sound signal that do not contain the noise signal. It is understandable that there are many processes for denoising sound signals based on sound features, and this application does not limit this.

[0047] Similarly, in this embodiment and another embodiment, when the audio detection module determines that the characteristics of each sound signal do not match the characteristics of the noise signal, it determines that there is no noise signal in the first initial sound signal and the second initial sound signal, and directly determines the first initial sound signal and the second initial sound signal as the first target sound signal and the second target sound signal.

[0048] In this way, after screening out the target speaker, the intelligent conference system can further collect the target sound signal generated by the target speaker during the speech through multiple audio collection units in the audio detection module, and identify the accuracy of the identified target speaker based on the target sound signal.

[0049] In a feasible implementation manner, the step of "determining the speaker verification result corresponding to the target speaker according to each of the target sound signals" in the above step S20 may specifically include steps S203 to S207: Step S203: determining a first signal receiving time corresponding to the first target sound signal and a second signal receiving time corresponding to the second target sound signal; Step S204: determining the sound source location information according to the first signal receiving time and the second signal receiving time; Step S205: determining the speaker position information of the target speaker based on the first image data, and comparing the sound source position information with the speaker position information to obtain a comparison result; Step S206: when it is detected that the comparison result is that the sound source position information matches the speaker position information, determining that the speaker verification result is verification passed; Step S207: when it is detected that the comparison result is that the sound source position information and the speaker position information do not match, determining that the speaker verification result is a verification failure.

[0050] In this embodiment, after acquiring the first target sound signal and the second target sound signal, the audio detection module inputs the acquired first target sound signal and the second target sound signal into the above-mentioned data processing module. The data processing module first determines the first signal receiving time corresponding to the first target sound signal, and determines the second signal receiving time corresponding to the second target sound signal. Then, the data processing module performs a difference calculation between the first signal receiving time and the second signal receiving time to obtain a receiving time difference, and determines the sound source position information corresponding to the target speaker based on the receiving time difference. Then, the data processing module determines the speaker position information of the target speaker based on the above-mentioned first image data, and compares the sound source position information with the speaker position information to obtain a comparison result. Finally, if the data processing module determines that the sound source position information matches the speaker position information, it determines that the target speaker detected based on the first image data is correct, and the data processing module thereby determines that the speaker verification result corresponding to the target speaker is verification passed. Similarly, if the data processing module determines that the sound source position information does not match the speaker position information, it determines that the target speaker identified based on the first image data is wrong, and the data processing module thereby determines that the speaker verification result corresponding to the target speaker is verification failed.

[0051] Exemplarily, for example, after the audio detection module obtains the above-mentioned first target sound signal and the second target sound signal, it first inputs the first target sound signal and the second target sound signal into the above-mentioned data processing module. The data processing module first extracts and determines the first signal receiving time when the above-mentioned first microphone collects the first target sound signal, and determines the second signal receiving time when the second microphone collects the second target sound signal. After that, the data processing module calculates the difference between the obtained first signal receiving time and the second signal receiving time to obtain the receiving time difference. At the same time, the data processing module reads the above-mentioned storage module to obtain the preset sound signal transmission speed. The data processing module further calculates the sound source position information matching the first target sound source signal and the second target sound signal based on the receiving time difference and the sound signal transmission speed. After that, the data processing module The data processing module processes the above-mentioned first image data, thereby determining the speaker position information corresponding to the above-mentioned target speaker at the conference site based on the first image data, and the data processing module then compares the acquired speaker position information with the sound source position information to obtain a comparison result. Finally, if the data processing module determines that the comparison result is that the speaker position information and the sound source position information match, it is determined that the target speaker detected based on the first image data is consistent with the target speaker who sends the target sound signal, and the data processing module thereby determines that the speaker verification result corresponding to the target speaker is verification passed. Similarly, if the data processing module determines that the sound source position information and the speaker position information do not match, it is determined that there is an error in the target speaker identified based on the first image data, and the data processing module thereby determines that the speaker verification result corresponding to the target speaker is verification failed.

[0052] In this way, after screening out the target speaker, the intelligent conference system can further collect the target sound signal generated by the target speaker during the speech through multiple audio collection units in the audio detection module, and identify the accuracy of the identified target speaker based on the target sound signal.

[0053] Step S30: when it is detected that the speaker verification result is passed, capturing second image data containing the target speaker through the image detection module, wherein the target speaker is located at the center of the second image data; In this embodiment, when the intelligent conference system detects that the speaker verification result is passed, it further controls the above-mentioned image detection module, so that the image detection module takes a picture of the conference scene to capture the second image data containing the target speaker and the target speaker is at the center of the image.

[0054] Exemplarily, for example, when the data processing unit detects that the speaker verification result is passed, the data processing unit uploads the speaker verification result to the intelligent conference system. The intelligent conference system determines the target camera device that can capture the target speaker from the multiple cameras configured in the image detection module based on the above-mentioned speaker position information, and based on the above-mentioned speaker position information and the camera device adjustment parameters generated to match the target camera device, the intelligent conference system then controls the image detection module, and the image detection module adjusts the target camera device according to the camera device adjustment parameters, and controls the target camera device to shoot the target speaker, thereby capturing the second image data that includes the target speaker and the target speaker is in the center of the image.

[0055] In this way, the intelligent conference system can, when determining that the target speaker has been accurately identified, further call the camera device to capture second image data containing the target speaker with the target speaker at the center.

[0056] Step S40: Generate a first conference video based on the second image data and each of the target sound signals, and send the first conference video to a preset target conference interface to display the first conference video to other online participants through the target conference interface.

[0057] In this embodiment, after capturing the above-mentioned second image data, the image detection module uploads the second image data to the intelligent conference system. At this time, the intelligent conference system fuses the second image data with the above-mentioned target sound signals to generate a first conference video, and sends the first conference video to a target conference interface that matches the conference site, thereby displaying the first conference video on the target conference interface, so that other online participants in the target conference interface can determine the target speaker through the first conference video and view the speech content of the target speaker.

[0058] Exemplarily, for example, after capturing the above-mentioned second image data, the image detection module further uploads the second image data to the intelligent conference system. At this time, the intelligent conference system fuses the second image data with the above-mentioned target sound signals to generate a first conference video. The intelligent conference system then sends the first conference video to a preset target conference interface, so that other online participants in the target conference interface can view the target speaker through the target conference interface and understand the speech content of the target speaker.

[0059] In this embodiment, during operation, the intelligent conference system first controls the image detection module configured in itself, so that the image detection module calls multiple cameras included in the scene to shoot the conference scene to obtain multiple image data to be fused, and the image detection module then fuses the multiple image data to be fused to obtain the first image data containing the faces of all participants. The image detection module sends the first image data to the data processing module configured in the intelligent conference system, and the data processing module screens multiple participants based on the first image data to determine the target speaker who is speaking. After that, the intelligent conference system controls the audio detection module configured in itself, so that the audio detection module calls multiple audio detection units configured at the conference scene to collect each target sound signal generated by the target speaker when speaking at the conference scene. The audio detection module then inputs each target sound signal into the data processing unit, and the data The processing unit determines whether the screened target speaker is correct based on each target sound signal, and generates a corresponding speaker verification result. After that, when the intelligent conference system detects that the speaker verification result is verified, it further controls the above-mentioned image detection module, so that the image detection module shoots the conference scene to capture the second image data containing the target speaker and the target speaker is at the center of the image. Finally, the image detection module uploads the second image data to the intelligent conference system. At this time, the intelligent conference system fuses the second image data with the above-mentioned target sound signals to generate a first conference video, and sends the first conference video to the target conference interface that matches the conference scene, so as to display the first conference video on the target conference interface, so that other online participants in the target conference interface can determine the target speaker through the first conference video and view the speech content of the target speaker.

[0060] In this way, the present application solves the technical problem of low accuracy of the judgment result of the current speaker by the intelligent conference system in the related technology, that is, the present application identifies multiple participants by calling the image detection module to determine the target speaker who is speaking, and determines whether the detected target speaker is accurate through the audio detection module, so that when it is determined that the detected target speaker is accurate, the image data containing the target speaker and the collected target sound signal are fused to generate a conference video, and the conference video is sent to the conference interface, so that the intelligent conference system can identify the location information of the target speaker who is speaking according to the sound signal during operation, and thus determine whether the identified speaker is accurate according to the location information, thereby enabling the intelligent conference system to accurately identify the target speaker at the conference site when the on-site environment changes significantly, thereby achieving the technical effect of improving the accuracy of the judgment result of the intelligent conference system on the current speaker, and greatly improving the efficiency of the meeting.

[0061] Based on the first embodiment of the present application, a second embodiment of the present application is proposed. In the second embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above description and will not be described in detail later. Figure 3 , Figure 3 The flowchart provided in the second embodiment of the scene perception and interaction method of the intelligent video conference of the present application is as follows: Figure 3 As shown, after the above step S40, the scene perception and interaction method of the intelligent video conference of the present application may further include steps A10 to A50: Step A10: capturing a plurality of third image data including the target speaker through the image detection module; Step A20: determining the moving trajectory of the target speaker based on the plurality of third image data, and determining the initial shooting angle of the image detection module; Step A30: generating an angle adjustment parameter based on the movement trajectory and the initial shooting angle, and adjusting the initial shooting angle to a target shooting angle according to the angle adjustment parameter; Step A40: controlling the image detection module to capture fourth image data including the target speaker according to the target shooting angle; Step A50: Generate a second conference video based on the fourth image data and each of the target sound signals, and send the second conference video to the target conference interface to display the second conference video to the other online participants through the target conference interface.

[0062] In this embodiment, after sending the above-mentioned first conference video to the above-mentioned target conference interface, the intelligent conference system further calls the above-mentioned image detection module to shoot the target speaker to capture multiple third image data containing the target speaker. Afterwards, the image detection module inputs the captured multiple third image data into the above-mentioned data processing unit, and the data processing unit extracts the image features contained in each of the multiple third image data, and identifies the position change of the target speaker based on each image feature, thereby determining the movement trajectory of the target speaker at the conference site based on the position change. At the same time, the data processing unit extracts the initial shooting angle of the above-mentioned target camera device, and then the data processing unit determines the angle adjustment parameter based on the movement trajectory and the initial shooting angle. , and sends the angle adjustment parameters to the image detection module, which adjusts the target camera device from the initial shooting angle to the target shooting angle according to the angle adjustment parameters. Then, the image detection module controls the target shooting device to shoot the target speaker, thereby capturing fourth image data containing the target speaker and with the target speaker at the center of the image. Finally, the image detection module uploads the acquired fourth image data to the intelligent conference system, which fuses the fourth image data with the above-mentioned target sound signal to generate a second conference video. The intelligent conference system then sends the second conference video to the preset target conference interface, so that the target speaker can always be in the center of the conference video watched by other online participants during the movement.

[0063] Exemplarily, for example, after the intelligent conference system sends the above-mentioned first conference video to the above-mentioned target conference interface, the intelligent conference system further calls the image detection module, so that the image detection module calls the above-mentioned camera device A, camera device B, and camera device C to simultaneously shoot the target speaker to capture multiple third image data containing the target speaker. Afterwards, the image detection module inputs the multiple third image data into the data processing unit, and the data processing unit extracts the image features contained in each of the multiple third image data, and determines the position change of the target speaker photographed by the camera device A, camera device B, and camera device C respectively according to the image features, and determines the movement trajectory of the target speaker according to the position change of the speaker photographed by the camera device A, camera device B, and camera device C respectively. At the same time, the image detection module determines the camera device A, camera device B, and camera device C that can capture the front of the target speaker. The target camera is camera device A, and the image detection module detects the initial shooting angle corresponding to camera device A. Thereafter, the image detection module determines the angle adjustment parameters used when adjusting camera device A according to the initial shooting angle and the moving trajectory. The image detection module adjusts the shooting angle of camera device A according to the angle adjustment parameters, thereby adjusting the shooting angle of camera device A from the initial shooting angle to the target shooting angle matching the angle adjustment parameters. Then, the image detection module controls camera device A to shoot the target speaker to capture the fourth image data containing the target speaker and the target speaker being in the center of the image. Finally, the image detection module fuses the fourth image data with the above-mentioned target sound signals to generate a second conference video, and sends the second conference video to the preset target conference interface, so that the target speaker can always be in the center position of the conference video watched by other online participants during the movement.

[0064] In this way, the intelligent conference system avoids the situation where the target speaker cannot be displayed in the conference video in the conference interface when the target speaker moves out of the detection range of the camera device. That is, the intelligent conference system can detect the position of the target speaker during the meeting, and when the movement of the target speaker is detected, the camera can be adjusted in time to capture image data containing the target speaker, thereby ensuring that even when the target speaker is in a moving state, the target speaker is always in the center of the picture seen by other online participants in the target conference interface.

[0065] Based on the first embodiment and / or the second embodiment of the present application, the third embodiment of the present application is proposed here. In the third embodiment of the present application, the same or similar contents as the above embodiments can be referred to the above introduction, and will not be repeated later. On this basis, after the above step S40, the scene perception and interaction method of the intelligent video conference of the present application can also include steps B10~B30: Step B10: determining the identity information of the target speaker; Step B20: updating the first conference video based on the identity information to obtain a third conference video; Step B30: Send the third conference video to the target conference interface to display the third conference video to the other online participants through the target conference interface.

[0066] It should be noted that the identity identification information is identification information that can indicate the identity of the target speaker, and specifically may be one or more identification information such as name, rank, representative group, etc.

[0067] In this embodiment, after uploading the first conference video to the target conference interface, the intelligent conference system further identifies the identity information of the target speaker based on the second image data. Thereafter, the intelligent conference system merges the identity information with the first conference video to generate a third conference video. Finally, the intelligent conference system sends the generated third conference video to the target conference interface to play the third conference video through the target conference interface, so that other participants can view the speech content of the target speaker through the third conference video and determine the identity information of the target speaker.

[0068] Exemplarily, for example, after uploading the first conference video to the target conference interface, the intelligent conference system further identifies the name information and position information of the target speaker based on the above-mentioned second image data. Thereafter, the intelligent conference system adds the acquired name information and position information to the first conference video to generate a third conference video. Finally, the intelligent conference screen uploads the generated third conference video to the above-mentioned target conference screen to update the first conference video being played in the target conference interface, so that other participants can determine the identity of the target speaker through the third conference video and hear the content of the target speaker's speech.

[0069] In this way, the intelligent conference system can identify the identity information of the target speaker based on the second image data, and mark the identity information in the conference video, so that other online participants in the target conference interface can understand the identity information of the target speaker based on the conference video, thereby further enhancing the conference experience.

[0070] In a feasible implementation manner, the above step B10 may specifically include steps B101 to B103: Step B101: determining a plurality of preset personnel characteristics and a plurality of preset identity identifiers that match the preset personnel characteristics; Step B102: based on the image features included in the first image data, screening the plurality of preset personnel features to determine the target personnel features matching the image features; Step B103: determining the preset identity identification that matches the target person's characteristics as the identity identification information of the target speaker.

[0071] Exemplarily, for example, after uploading the first conference video to the target conference interface, the intelligent conference system first reads the storage module configured by itself to obtain a name mapping table containing multiple preset personnel features and preset name information that matches each of the multiple preset personnel features. After that, the intelligent conference system queries the name mapping table based on the image features of the extracted second image data to compare the image features with the multiple preset personnel features to obtain multiple comparison results. The intelligent conference system then determines the target personnel features that match the image features based on the multiple comparison results. Finally, the intelligent conference determines the preset name information corresponding to the target personnel features in the name mapping table as the name information corresponding to the target speaker. Similarly, the intelligent conference system can also obtain a position mapping table containing multiple preset personnel features and preset position information that matches each of the multiple preset personnel features. Afterwards, the intelligent conference system queries the name mapping table based on the image features extracted from the second image data to compare the image features with the multiple preset personnel features to obtain multiple comparison results. The intelligent conference system then determines the target personnel features that match the image features based on the multiple comparison results. Finally, the intelligent conference determines the preset position information corresponding to the target personnel features in the position mapping table as the position information corresponding to the target speaker.

[0072] In this way, the intelligent conference system can identify the identity information of the target speaker based on the second image data, and mark the identity information in the conference video, so that other online participants in the target conference interface can understand the identity information of the target speaker based on the conference video, thereby further enhancing the conference experience.

[0073] Based on the various embodiments of the present application, a fourth embodiment of the present application is proposed here. In the fourth embodiment of the present application, the same or similar contents as the above embodiments can be referred to the above introduction, and will not be repeated later. On this basis, after the above step S40, the scene perception and interaction method of the intelligent video conference of the present application can also include steps C10~C30: Step C10: receiving an interactive voice signal triggered by the target speaker, and determining a target interactive instruction according to the interactive voice signal; Step C20: when it is detected that the instruction type of the target interaction instruction is an environment control instruction, determining the target environment adjustment device corresponding to the conference site based on the environment control instruction; Step C30: Control the target environment adjustment device according to the environment control instruction, so as to adjust the environment parameters of the conference site through the target environment adjustment device.

[0074] In this embodiment, after sending the above-mentioned first conference video to the above-mentioned target conference interface, the intelligent conference system further calls the audio detection module to receive the interactive voice signal emitted by the target speaker, and the audio detection module parses the voice interaction signal to determine the target interaction instruction corresponding to the voice interaction signal. After that, the intelligent conference system detects the instruction type of the target interaction instruction, and when it is detected that the instruction type is an environmental control instruction for changing the conference site environment, the target environment adjustment device for adjusting the conference site environment is determined based on the environmental control instruction. Finally, the intelligent conference system controls the target environment adjustment device according to the environmental control instruction to modify the environmental parameters of the conference site through the target environment adjustment device.

[0075] Exemplarily, for example, when the target speaker needs to turn off the lights at the conference site during his speech, the target speaker can send an interactive voice signal "turn off the lights at the conference site". At this time, the intelligent conference system controls the audio detection module, and the audio detection module calls the first microphone or the second microphone to collect the interactive voice signal "turn off the lights at the conference site" triggered by the target speaker, and the audio detection module queries the preset signal mapping table containing multiple preset voice signals and preset interactive instructions that match the multiple preset voice signals based on the interactive voice signal, thereby determining that the target interactive instruction that matches the interactive voice signal "turn off the lights at the conference site" is the lights off. After that, the intelligent conference system determines that the lights off instruction is an instruction for adjusting the environment of the conference site, thereby determining that the instruction type of the lights off instruction is an environment adjustment instruction. The intelligent conference system thus determines that based on the lights off instruction, the light adjustment device configured at the conference site is the target environment adjustment device for turning off the lights at the conference site. Finally, the intelligent conference system adjusts the light adjustment device according to the lights off instruction to turn off the lights at the conference site through the light adjustment device.

[0076] In this way, the intelligent conference system can analyze the voice signals sent by the speaker during the meeting to determine the control instructions that the speaker wants to execute, and adjust the environment of the conference site according to the control instructions.

[0077] In addition, in this embodiment and another embodiment, after controlling the target environment adjustment device, the intelligent conference system can also call the light detection module configured by itself to detect the conference site, so as to determine the real-time ambient light parameters of the conference site. After that, the intelligent conference system reads the storage module configured by itself to obtain an ambient light parameter mapping table containing multiple preset ambient light parameters and preset exposure parameters that match each of the multiple preset ambient light parameters, and queries the ambient light parameter mapping table according to the real-time ambient light parameters to determine the target ambient light parameters that match the actual ambient light parameters. The intelligent conference system then determines the preset exposure parameters corresponding to the target ambient light parameters in the ambient light parameter mapping table as the target exposure parameters that match the image acquisition unit, and the intelligent conference system then adjusts the exposure parameters of the target image detection unit according to the target exposure parameters. In this way, the intelligent conference system can adjust the exposure parameters of the image detection module according to the ambient light of the conference site to ensure that the image detection module captures clear image data.

[0078] For example, in order to help understand the implementation process of the scene perception and interaction method of the intelligent video conference obtained by combining this embodiment with the above embodiments, please refer to Figure 4 , Figure 4 A brief flow chart of the scene perception and interaction method of the intelligent video conference in this application is provided, specifically: In this embodiment, during operation, the intelligent conference system first calls the camera device configured at the conference site through the image detection module configured by itself to shoot the conference site to capture the first image data containing multiple participants. The image detection module then processes the first image data to determine the target speaker who is speaking among the multiple participants based on the image features of the first image data. At the same time, the intelligent conference system calls the first detection unit (i.e., the first microphone mentioned above) configured at the conference site to collect the first target sound signal generated at the conference site, and calls the second detection unit (i.e., the second microphone mentioned above) configured at the conference site to collect the second target sound signal generated at the conference site through the audio detection module configured by itself. The audio detection module then determines the sound source position information based on the first target sound signal and the second target sound signal, and uploads the sound source position information to the intelligent conference system. At this time, the intelligent conference system determines the speaker position information of the target speaker at the conference site based on the first image data. Afterwards, the intelligent conference system compares the speaker position information with the sound source position information to determine whether the speaker position information and the sound source position information match. If the intelligent conference system determines that the speaker position information and the sound source position information match, it determines that the identified target speaker is correct, and calls the above-mentioned camera device to shoot the target speaker to capture the second image data containing the target speaker and the target speaker is at the center of the image. The intelligent conference system then fuses the second image data with the above-mentioned target sound signals to generate a first conference video. The intelligent conference system uploads the first conference video to the target conference interface to display the first conference video to other online participants through the target conference interface. Afterwards, the intelligent conference system uses the image detection module to call the camera device to detect the conference site to identify the movement trajectory of the target speaker. The intelligent conference system adjusts the shooting angle of the camera device according to the movement trajectory, so that the camera device shoots the target speaker according to the adjusted shooting angle to capture fourth image data containing the target speaker. The intelligent conference system then fuses the fourth image data with the above-mentioned target sound signals to generate a second conference video, and uploads the second conference video to the target conference interface to display the second conference video to other online participants through the target conference interface, ensuring that the target speaker can always be in the center of the picture; Then, the intelligent conference system processes the collected fourth image data to extract image features of the fourth image data, and determines the identity information of the target speaker based on the image features of the fourth image data. The intelligent conference system adds the identity information to the second conference video to generate a third conference video, and uploads the third conference video to the target conference interface to display the third conference video to other online participants through the target conference interface, so that other online participants can understand the identity information of the target speaker while watching the conference screen; Finally, if the target speaker needs to adjust the environmental parameters of the conference site during the speech, the target speaker can send an interactive voice signal. At this time, the intelligent conference system controls the audio detection module, and the audio detection module calls the above-mentioned detection units to collect the interactive voice signal triggered by the target speaker, and determines the target interaction instruction triggered by the target speaker based on the interactive voice signal. The intelligent conference system then determines the target environment adjustment device for adjusting the conference site according to the target interaction instruction, and controls the target environment adjustment device according to the target interaction instruction to achieve adjustment of the conference site environment.

[0079] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the scene perception and interaction methods of the intelligent video conferencing of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0080] The present application provides an intelligent conference system, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the scene perception and interaction method of the intelligent video conference in the above-mentioned embodiment one.

[0081] Reference below Figure 5 , which shows a schematic diagram of the structure of an intelligent conference system suitable for implementing the embodiment of the present application. The intelligent conference system in the embodiment of the present application may include, but is not limited to, an intelligent conference system configured with an image detection module, an audio detection module, a data processing module, or a mobile terminal, a data storage control terminal, a PC, and other terminals connected to an electronic control unit supporting the intelligent conference system. Figure 5 The intelligent conference system shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0082] like Figure 5 As shown, the intelligent conference system may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the intelligent conference system are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the intelligent conference system to communicate with other devices wirelessly or wired to exchange data. Although the figure shows an intelligent conference system with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0083] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0084] The intelligent conference system provided by the present application adopts the scene perception and interaction method of the intelligent video conference in the above-mentioned embodiment, which can solve the technical problem of low accuracy of the judgment result of the current speaker by the intelligent conference system in the related technology. Compared with the prior art, the beneficial effects of the intelligent conference system provided by the present application are the same as the beneficial effects of the scene perception and interaction method of the intelligent video conference provided by the above-mentioned embodiment, and the other technical features of the intelligent conference system are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0085] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0086] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0087] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the scene perception and interaction method of the intelligent video conferencing in the above-mentioned embodiment.

[0088] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0089] The computer-readable storage medium may be included in the intelligent conference system; or may exist independently without being assembled into the intelligent conference system.

[0090] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the intelligent conference system, the intelligent conference system: uses the image detection module to shoot the conference scene to capture the first image data containing multiple participants, and screens the multiple participants according to the first image data to determine the target speaker; obtains each target sound signal contained in the conference scene through the audio detection module, and determines the speaker verification result corresponding to the target speaker according to each target sound signal; when it is detected that the speaker verification result is verified, captures the second image data containing the target speaker through the image detection module, wherein the target speaker is at the center position of the second image data; generates a first conference video based on the second image data and each target sound signal, and sends the first conference video to a preset target conference interface, so as to display the first conference video to other online participants through the target conference interface.

[0091] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0094] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the scene perception and interaction method of the above-mentioned intelligent video conference, and can solve the technical problem of low accuracy of the judgment result of the current speaker by the intelligent conference system in the related art. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the scene perception and interaction method of the intelligent video conference provided by the above-mentioned embodiment, and will not be repeated here.

[0095] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the scene perception and interaction method for intelligent video conferencing as described above.

[0096] The computer program product provided by the present application can solve the technical problem in the related art that the accuracy of the judgment result of the current speaker by the intelligent conference system is low. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the scene perception and interaction method of the intelligent video conference provided by the above embodiment, which will not be repeated here.

[0097] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A scene perception and interaction method for intelligent video conferencing, characterized in that: The scene perception and interaction method of the intelligent video conference is applied to an intelligent conference system, wherein the intelligent conference system comprises an image detection module and an audio detection module, and the scene perception and interaction method of the intelligent video conference comprises: The conference site is photographed by the image detection module to capture first image data containing a plurality of conference participants, and the plurality of conference participants are screened according to the first image data to determine a target speaker; Acquire each target sound signal contained in the conference site through the audio detection module, and determine the speaker verification result corresponding to the target speaker according to each target sound signal; When it is detected that the speaker verification result is passed, capturing second image data containing the target speaker through the image detection module, wherein the target speaker is located at the center of the second image data; A first conference video is generated based on the second image data and each of the target sound signals, and the first conference video is sent to a preset target conference interface so as to display the first conference video to other online participants through the target conference interface.

2. The scene perception and interaction method of intelligent video conferencing according to claim 1, characterized in that: The step of screening the plurality of conference participants according to the first image data to determine a target speaker comprises: extracting image features contained in the first image data; Determine facial features of the multiple conference participants that match each other based on the image features, and determine facial movement features included in the facial features of each of the participants; The conference participant corresponding to the facial movement feature is determined as the target speaker.

3. The scene perception and interaction method of intelligent video conferencing according to claim 1, characterized in that: The audio detection module includes a first detection unit and a second detection unit, and the step of obtaining each target sound signal contained in the conference site through the audio detection module includes: The first detection unit collects a first initial sound signal contained in the conference site, and the second detection unit collects a second initial sound signal contained in the conference site; The first initial sound signal and the second initial sound signal are subjected to noise reduction processing to obtain a first target sound signal and a second target sound signal.

4. The scene perception and interaction method of intelligent video conferencing according to claim 3, characterized in that: The step of determining the speaker verification result corresponding to the target speaker according to each of the target sound signals comprises: Determining a first signal reception time corresponding to the first target sound signal and a second signal reception time corresponding to the second target sound signal; Determine the sound source location information according to the first signal receiving time and the second signal receiving time; Determine the speaker position information of the target speaker based on the first image data, and compare the sound source position information with the speaker position information to obtain a comparison result; In the case where it is detected that the comparison result is that the sound source position information matches the speaker position information, determining that the speaker verification result is verification passed; When it is detected that the comparison result is that the sound source position information and the speaker position information do not match, it is determined that the speaker verification result is a verification failure.

5. The scene perception and interaction method of intelligent video conferencing according to claim 1, characterized in that: After the step of sending the first conference video to a preset target conference interface, the method further includes: Capturing a plurality of third image data including the target speaker through the image detection module; Determine the moving trajectory of the target speaker based on the plurality of third image data, and determine the initial shooting angle of the image detection module; generating an angle adjustment parameter based on the movement trajectory and the initial shooting angle, and adjusting the initial shooting angle to a target shooting angle according to the angle adjustment parameter; Controlling the image detection module to capture fourth image data including the target speaker according to the target shooting angle; A second conference video is generated based on the fourth image data and each of the target sound signals, and the second conference video is sent to the target conference interface so as to display the second conference video to the other online participants through the target conference interface.

6. The scene perception and interaction method of intelligent video conferencing according to claim 1, characterized in that: After the step of sending the first conference video to a preset target conference interface, the method further includes: Determining the identity information of the target speaker; Based on the identity information, the first conference video is updated to obtain a third conference video; The third conference video is sent to the target conference interface so as to display the third conference video to the other online participants through the target conference interface.

7. The scene perception and interaction method of intelligent video conferencing according to claim 1, characterized in that: After the step of sending the first conference video to a preset target conference interface, the method further includes: Receiving an interactive voice signal triggered by the target speaker, and determining a target interactive instruction according to the interactive voice signal; In the case where it is detected that the instruction type of the target interaction instruction is an environment control instruction, determining a target environment adjustment device corresponding to the conference site based on the environment control instruction; The target environment adjustment device is controlled according to the environment control instruction, so as to adjust the environmental parameters of the conference site through the target environment adjustment device.

8. An intelligent conference system, characterized in that: The system includes: an image detection module, an audio detection module, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the scene perception and interaction method for intelligent video conferencing as described in any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the scene perception and interaction method of the intelligent video conferencing as described in any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the scene perception and interaction method for intelligent video conferencing as described in any one of claims 1 to 7 are implemented.

Images