Target positioning method and device, video conference equipment and storage medium

Abstract

The invention discloses a target positioning method and device, video conference equipment and a storage medium, and relates to the technical field of target positioning, and the method comprises the steps: obtaining the position information of human voice; adjusting an image acquired by a lens according to the position information; acquiring a plurality of lip key points of at least one face image of the image, and processing the plurality of lip key points based on a pre-trained lip movement recognition model to determine a lip state of the face image; wherein the lip state comprises a moving state and a static state; when it is determined that the lip state of the at least one face image is a motion state, determining that at least one spokesman exists in the image, and controlling a lens to obtain a close-up picture of the spokesman; the objective of the invention is to solve the technical problem that the existing tracking camera is low in spokesman recognition precision.

Description

Technical Field

[0001] The present application relates to the field of target positioning technology, and in particular to a target positioning method, device, video conferencing equipment and storage medium. Background Art

[0002] Under current technical conditions, in order to achieve a better conference experience, tracking cameras are usually introduced and sound source localization technology is used to determine the position of the current speaker in the video conference. The tracking camera can capture and output close-up shots of the speaker. In this way, participants can observe the speaker's facial expressions and subtle movements in detail, which greatly improves the conference experience and ensures the high quality and effectiveness of the video conference.

[0003] However, the current method of identifying speakers only determines the position of the current speaker through sound source localization technology. When there are many people gathered around the speaker, the tracking camera can easily obtain close-up shots of other people. Summary of the invention

[0004] The main purpose of the present application is to provide a target positioning method, device, video conferencing equipment and storage medium, aiming to solve the technical problem of low accuracy of speaker identification by existing tracking cameras.

[0005] To achieve the above object, the present application proposes a target positioning method, which is applied to a target positioning device, wherein the target positioning device includes at least one lens, and the method includes: Get the location information of human voice; adjusting the image acquired by the lens according to the position information; Acquire multiple lip key points of at least one face image of the image, and process the multiple lip key points based on a pre-trained lip movement recognition model to determine the lip state of the face image; wherein the lip state includes a moving state and a static state; When it is determined that the lip state of at least one face image is in a moving state, it is determined that there is at least one speaker in the image, and the camera is controlled to obtain a close-up image of the speaker.

[0006] In one embodiment, the lip key points include upper inner lip key points and lower inner lip key points, and the training method of the lip movement recognition model includes: Constructing a first one-dimensional feature input according to the distances between a plurality of upper inner lip key points and a plurality of lower inner lip key points; The pre-trained model is trained according to the first one-dimensional feature input to obtain the lip movement recognition model.

[0007] In one embodiment, the lip key points further include an upper outer lip key point and a lower outer lip key point, and after constructing the first one-dimensional feature input according to the distances between the plurality of upper inner lip key points and the plurality of lower inner lip key points, the step further includes: Constructing a second one-dimensional feature input according to the distances between a plurality of upper outer lip key points and a plurality of lower inner lip key points; and / or, constructing a third one-dimensional feature input according to distances between a plurality of lower outer lip key points and a plurality of upper inner lip key points; And / or, constructing a fourth one-dimensional feature input according to distances between a plurality of upper outer lip key points and a plurality of lower outer lip key points.

[0008] In one embodiment, the training method of the lip movement recognition model includes: Establishing a coordinate relationship between a plurality of lip key points; Constructing a two-dimensional feature input according to the coordinates of the plurality of lip key points; The pre-trained model is trained according to the two-dimensional feature input to obtain the lip movement recognition model.

[0009] In one embodiment, the acquiring of a plurality of lip key points of at least one face image of the image comprises: Processing the image to determine a face image in the image; Processing the facial image to determine a plurality of facial key points of the facial image; Filter out multiple lip key points from multiple face key points.

[0010] In one embodiment, the lens includes a panoramic lens, and adjusting the image acquired by the lens according to the position information includes: Controlling the panoramic lens to acquire images; Extracting an image corresponding to the position information from the image; The extracted image is processed to obtain a face image in the extracted image.

[0011] In one embodiment, the lens further includes a close-up shot, and adjusting the image acquired by the lens according to the position information further includes: Determining the distance between the target positioning device and the sound source position of the human voice according to the position of the target positioning device and the position information; When it is determined that the distance between the target positioning device and the sound source of the human voice is greater than a preset distance, adjusting the rotation direction of the close-up lens according to the position information; The magnification of the close-up camera is adjusted so that the close-up camera can recognize a human face image.

[0012] In addition, to achieve the above-mentioned purpose, the present application also proposes a target positioning device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the target positioning method as described in any one of the above items; and also includes a microphone array for obtaining the location information of a human voice; and at least one lens for obtaining an image.

[0013] In addition, to achieve the above objectives, the present application also proposes a video conferencing device, which includes the target positioning device as described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the target positioning method described above are implemented.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: The present application obtains the position information of the human voice; adjusts the image obtained by the lens according to the position information; obtains multiple lip key points of at least one face image of the image, and processes the multiple lip key points based on a pre-trained lip movement recognition model to determine the lip state of the face in the image; wherein the lip state includes a moving state and a static state; when the lip state is determined to be a moving state, it is determined that there is at least one speaker in the image, and the lens is controlled to obtain a close-up image of the speaker. With such a setting, in actual application, when the face images of multiple people are obtained, the present application can identify the person who is speaking among the multiple people according to the lip states of the multiple people, thereby obtaining a close-up image of the speaker. Compared with the existing method of determining the position of the speaker only by sound source localization technology, the target localization method of the present application is more accurate in identifying the speaker, and can also accurately obtain a close-up image of the speaker in scenes with a large crowd density. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 A schematic diagram of a process flow provided for Embodiment 1 of the present application; Figure 2 A schematic diagram of the process provided for Example 2 of the present application; Figure 3 A schematic diagram of the process provided in Example 3 of the present application; Figure 4 A schematic diagram of a process flow provided for Embodiment 4 of the present application; Figure 5 A schematic diagram of the process provided for Embodiment 5 of the present application; Figure 6 A schematic diagram of a process flow provided for Embodiment 6 of the present application; Figure 7 This is a schematic diagram of multiple facial key points distributed on a face.

[0019] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0021] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0022] Under current technical conditions, in order to achieve a better conference experience, tracking cameras are usually introduced and sound source localization technology is used to determine the position of the current speaker in the video conference. The tracking camera can capture and output close-up shots of the speaker. In this way, participants can observe the speaker's facial expressions and subtle movements in detail, which greatly improves the conference experience and ensures the high quality and effectiveness of the video conference.

[0023] However, the current method of identifying speakers only determines the position of the current speaker through sound source localization technology. When there are many people gathered around the speaker, the tracking camera can easily obtain close-up shots of other people.

[0024] To this end, the present application provides a target positioning method, device, video conferencing equipment and storage medium, aiming to solve the technical problem of low accuracy of speaker identification by existing tracking cameras.

[0025] refer to Figure 1 In one embodiment of the present application, the present application proposes a target positioning method, which is applied to a target positioning device, wherein the target positioning device includes at least one lens, and the method includes: Step S100: Acquire the location information of the human voice; It should be noted that the target positioning device has a built-in human voice detection algorithm to process the received sound to determine whether the received sound is a human voice. For example, the algorithm first extracts features in the audio signal, such as spectrum features, cepstrum features, and linear prediction coding. Then, based on the extracted features, a classifier is trained using a large amount of labeled audio data to classify the extracted features, so that it can accurately identify whether the received sound is a human voice.

[0026] This application can determine the location information of human voice by setting up a microphone array and using the sound source localization technology based on TDOA (Time Difference Of Arrival). For example, by estimating the time difference between the sound arriving at two microphones, a set of equations for the sound source position coordinates is obtained. Then, the precise azimuth coordinates of the sound source can be obtained by solving the set of equations.

[0027] Step S200: adjusting the image acquired by the lens according to the position information; It should be noted that after determining the sound source position information of the human voice, the angle range formed by the vertical line of the sound source position and the lens position is calculated, and the lens is controlled to rotate toward the sound source position and take pictures according to the angle range, or the image corresponding to the sound source position is extracted from the acquired image to ensure that the lens can obtain an image containing all possible speakers.

[0028] Step S300: acquiring a plurality of lip key points of at least one face image of the image, and processing the plurality of lip key points based on a pre-trained lip movement recognition model to determine the lip state of the face in the image; wherein the lip state includes a moving state and a static state; It should be noted that the lip movement recognition model based on the pre-training of the present application needs to process multiple lip key points of at least one facial image of multiple consecutive frames (for example, 30 frames) to determine the lip state of the facial image. In this way, it is equivalent to identifying the lip state of the facial image every 33.33 milliseconds, so that the target positioning method of the present application can identify the position of the speaker in real time, thereby tracking the position of the speaker in real time.

[0029] Optionally, since a person's lips move constantly when they speak, the positions of multiple lip key points will change constantly. The lip movement recognition model can track the changes of multiple lip key points in multiple consecutive frames (for example, the displacement, acceleration or overall shape changes of multiple lip key points), and compare the changes with a preset threshold. When the changes are greater than the preset threshold, it is determined that the lip state is in motion, that is, the person corresponding to the face image is speaking; when the changes are less than the preset threshold, it is determined that the lip state is in a static state, that is, the person corresponding to the face image is not speaking at this time.

[0030] Optionally, since the lip state is in an alternating open-closed state when a person speaks, the lip movement recognition model can track multiple relative position changes between multiple lip key points of the upper lip and multiple lip key points of the lower lip in multiple consecutive frames. When most of the multiple relative position changes are greater than a preset value, the lip state is in an alternating open-closed state (movement state), indicating that the person corresponding to the facial image is speaking; when most of the multiple relative position changes are less than a preset value, the lip state is in a static state, indicating that the person corresponding to the facial image is not speaking.

[0031] Among them, reference Figure 2 , obtaining a plurality of lip key points of at least one face image of the image comprises: Step S310: Processing the image to determine a face image in the image; It should be noted that after obtaining the image of the sound source position, the face image in the image must be obtained first to determine the multiple lip key points of the face image. The present application can process the image obtained by the lens through a pre-trained yolov7 model or a Haar feature classifier or other face detection algorithm model to obtain at least one face in the image.

[0032] Step S311: Processing the facial image to determine a plurality of facial key points of the facial image; It should be noted that the reference Figure 7 , Figure 7 The 0~95 points shown are all facial key points. Multiple facial key points are distributed on the facial features. This application processes the facial image based on a pre-trained facial key point model to determine multiple facial key points of the facial image.

[0033] Step S312: Filter out multiple lip key points from multiple facial key points.

[0034] It should be noted that Figure 3 Points 76 to 95 shown are all lip key points, and multiple lip key points are distributed on the upper and lower inner lips and the upper and lower outer lips. In this way, the lip movement recognition model can process multiple lip key points to determine the relative change between the upper and lower inner lips and / or the relative position change between the upper and lower outer lips, which helps to determine the lip state of the face image.

[0035] Step S400: when it is determined that the lip state of at least one face image is in a moving state, it is determined that there is at least one speaker in the image, and the camera is controlled to obtain a close-up image of the speaker.

[0036] It should be noted that when there is only one speaker in the acquired image, the magnification of the lens can be adjusted to obtain a clear close-up image of the speaker; when there are multiple speakers in the image, the images of multiple speakers in the image can be cut out and the images of multiple speakers can be merged into an external matrix frame to obtain close-up images of multiple speakers.

[0037] Specifically, the present application obtains the position information of the human voice; adjusts the image obtained by the lens according to the position information; obtains multiple lip key points of at least one face image of the image, and processes the multiple lip key points based on the pre-trained lip movement recognition model to determine the lip state of the face in the image; wherein the lip state includes a moving state and a static state; when the lip state is determined to be a moving state, it is determined that there is at least one speaker in the image, and the lens is controlled to obtain a close-up image of the speaker. With such a setting, the present application can identify the person who is speaking among the multiple people according to the lip states of the multiple people when the face images of multiple people are obtained, thereby obtaining a close-up image of the speaker. Compared with the existing method of determining the position of the speaker only by sound source localization technology, the target positioning method of the present application has a higher accuracy in identifying the speaker, and can also accurately obtain a close-up image of the speaker in a scene with a large crowd density. In practical applications, for example, in a large conference room, multiple participants sit around a round table to discuss issues. The target positioning method of the present application can identify the participant who is speaking among the multiple participants and obtain a close-up image of the speaking participant. For another example, in a classroom, the teacher is standing on the podium giving a lecture, while the students are distributed in different positions in the classroom. The target positioning method of this application can identify the student who is speaking among multiple students and obtain a close-up image of the speaking student. For another example, in a large public speech event, the auditorium is full of listeners. The target positioning method of this application can identify the audience who is speaking among multiple audiences and obtain a close-up image of the speaking audience.

[0038] refer to Figure 3 In one embodiment of the present application, the lip key points include upper inner lip key points and lower inner lip key points, and the training method of the lip movement recognition model includes: Step S320: constructing a first one-dimensional feature input according to the distances between the multiple upper inner lip key points and the multiple lower inner lip key points; It should be noted that one-dimensional feature input in the lip movement recognition model refers to feature data constructed based on the distance or position relationship between the key points of the lips. It is used to describe the slight changes in the lips when speaking and serves as one of the important inputs for model training.

[0039] Step S321: training the pre-trained model according to the one-dimensional feature input to obtain the lip movement recognition model.

[0040] It should be noted that the change in the distance between multiple upper and lower inner lip key points can reflect the degree of lip opening. This application forms sequence features from the distances between multiple lip key points of multiple consecutive frames of facial images marked with motion state / static state, and processes the sequence features through a convolutional neural network (such as VarGFaceNet) or a recurrent neural network (such as LongShort-Term Memory or Gated Recurrent Unit) to construct a one-dimensional feature input of multiple lip key points, and trains the pre-trained model (existing face recognition model) according to the one-dimensional feature input to obtain the corresponding prediction results. The CrossEntropyLoss loss function can be used to perform binary classification training on the prediction results, which can measure the difference between the model prediction results and the true labels, thereby guiding the pre-trained model to learn effectively and train the pre-trained model into a lip movement recognition model. Among them, the categories of the two categories are motion state and static state.

[0041] It should be noted that lip movement is a process that changes continuously over time, and the sequence formed by the distances of lip key points in multiple frames of images can fully reflect the movement trajectory of the lips over a period of time.

[0042] In this embodiment, the lip key points further include an upper outer lip key point and a lower outer lip key point, and after constructing the first one-dimensional feature input according to the distances between the plurality of upper inner lip key points and the plurality of lower inner lip key points, the step further includes: Constructing a second one-dimensional feature input according to the distances between a plurality of upper outer lip key points and a plurality of lower inner lip key points; and / or, constructing a third one-dimensional feature input according to distances between a plurality of lower outer lip key points and a plurality of upper inner lip key points; And / or, constructing a fourth one-dimensional feature input according to distances between a plurality of upper outer lip key points and a plurality of lower outer lip key points.

[0043] Since the movements of the inner lip and the outer lip are interrelated, when the distance between the key points of the upper and lower inner lips is considered separately, the training samples and features of the model are relatively small, which may lead to a large prediction deviation of the model output, that is, it is impossible to recognize more lip states. Since the distance between the key points of the upper and lower inner lips can be considered at the same time, the pre-trained model can be trained based on the first one-dimensional feature input to the fourth one-dimensional feature input, which helps to reduce the prediction deviation of the model in the recognition process. And by introducing the distance between the key points of the upper and lower inner lips and the upper and lower outer lips to construct a one-dimensional feature input, some feature confusion can be reduced. For example, different lip movements may result in the same distance between the upper and lower lips, but the relative relationship between the inner lip and the outer lip is different, thereby improving the model's ability to distinguish different lip states. And the distance change between the key points of the upper and lower inner lips and the upper and lower outer lips has more lip feature dimensions. More feature dimensions mean that the model has more information to distinguish different lip states.

[0044] It should be noted that the reference Figure 7 The key points of the upper and lower inner lips are 88~95 points, and the key points of the upper and lower outer lips are 76~87 points.

[0045] refer to Figure 4 In one embodiment of the present application, the training method of the lip movement recognition model includes: Step S330: establishing a coordinate relationship between a plurality of lip key points; It should be noted that, for each selected lip key point, its coordinates in the image can be obtained by image analysis technology (such as calculation based on pixel coordinates). Then, the coordinate relationship between multiple lip key points is established by calculating mathematical relationships such as distance, slope, angle, etc. between two points. For example, the middle position of the lips is used as the origin, and the coordinate relationship of multiple lip key points is established based on the distance between multiple lip key points and the middle position of the lips.

[0046] Step S331: constructing a two-dimensional feature input according to the coordinates of the plurality of lip key points; It should be noted that the two-dimensional feature input is feature data constructed based on the coordinates of multiple lip key points, and is used to describe the shape, position and changes of the lips.

[0047] Step S332: training the pre-trained model according to the two-dimensional feature input to obtain the lip movement recognition model.

[0048] It is understandable that when using multi-point position coordinates to construct a two-dimensional feature input, the coordinate information of multiple points can provide rich lip morphology information. For example, the positional relationship of each key point on the lips on a two-dimensional plane (image plane), including the contour and curvature of the lips. By determining the coordinate relationship of each lip key point, the shape, size and relative position of the lips can be quantified, which can better reflect the state of the lips. For example, the coordinate relationship of the key points of the upper lip and the lower lip can reflect the degree of opening and closing of the lips, and the coordinate relationship of the lip key points at the edge of the lips and the lip key points at the center of the lips can reflect the shape of the lips. For another example, the slope of the line connecting adjacent lip key points is calculated to determine the curvature of the lip edge. In this way, compared with the one-dimensional feature input of the distance of multiple lip key points, the pre-trained model is trained based on the two-dimensional feature input of multiple lip key points, so that the model can learn more lip morphology features, which helps to improve the accuracy of model prediction.

[0049] It should be noted that the training process of step S332 is the same as that of step S321, and will not be described in detail here.

[0050] refer to Figure 5 In one embodiment of the present application, the lens includes a panoramic lens, and adjusting the image acquired by the lens according to the position information includes: Step S210: controlling the panoramic lens to acquire an image; Step S220: extracting an image corresponding to the position information from the image; It should be noted that after the panoramic image is acquired, the image area to be extracted is determined based on the pre-acquired position information (such as coordinate range, area identification, etc.). The extraction operation can be implemented through image processing algorithms (such as image cropping, image segmentation, etc.) to extract the part of the panoramic image corresponding to the target position information.

[0051] Step S230: Process the extracted image to obtain a face image in the extracted image.

[0052] It should be noted that the multiple extracted images can be processed by a pre-trained face detection algorithm model such as a yolov7 model or a Haar feature classifier to obtain at least one face image in the extracted image. In this way, when the lip movement recognition model determines that the lip state of at least one face image is in a moving state according to the face image, it determines that the person corresponding to the at least one face image is a speaker. It should be noted that the panoramic lens usually has a wide viewing angle and can capture more scene information than an ordinary lens. Therefore, the panoramic lens is suitable for scenes where the position of the speaker is relatively fixed or there are many speakers, such as scenes such as auditoriums, classrooms or large conference rooms, where there will be multiple people who may speak at any time. In the case of multiple people speaking at the same time or randomly, since the panoramic lens can basically obtain more scene information, it is only necessary to extract the image obtained by the panoramic lens according to the sound source position of the human voice to obtain a rough image of the speaker, without rotating according to the position of the speaker.

[0053] In this embodiment, reference Figure 6 , the lens also includes a close-up shot, and the adjusting the image acquired by the lens according to the position information also includes: Step S240: determining the distance between the target positioning device and the sound source position of the human voice according to the position of the target positioning device and the position information; Step S250: when it is determined that the distance between the target positioning device and the sound source of the human voice is greater than a preset distance, adjusting the rotation direction of the close-up lens according to the position information; Step S260: adjusting the magnification of the close-up camera so that the close-up camera can recognize a facial image.

[0054] It should be noted that when the distance between the sound source of the human voice and the target positioning device is greater than the preset distance, the panoramic lens cannot obtain a clear face photo, resulting in the inability to analyze the lip state of the face image in the subsequent process. Therefore, when the distance between the sound source and the panoramic lens is greater than the preset distance, the close-up camera is controlled to rotate toward the sound source of the human voice, and the magnification of the close-up camera is adjusted so that the close-up camera can obtain a clear face image, and the close-up camera stops zooming in after the size of the face image is equal to the preset size. In this way, the lip movement recognition model analyzes the lip state of the face image obtained by the panoramic lens, and also needs to process the lip state of the face image obtained by the close-up camera to determine the lip state of the face image. With such a setting, in practical applications, the target positioning method of the present application can obtain close-up images of speakers at a long distance.

[0055] It is understandable that since the panoramic lens obtains the close-up picture of the speaker by cutting out the image, the close-up picture obtained by the panoramic lens may be jerky when the speaker moves. In this regard, the lip movement recognition model determines the existence of a speaker after processing the lip state of the face image obtained by the panoramic lens, and can control the close-up camera to obtain the close-up picture of the speaker and track the position of the speaker by rotating. It should be noted that in this case, since the lip movement recognition model has already processed the face image obtained by the panoramic lens and identified the speaker, the lip movement recognition model does not need to process the image obtained by the close-up.

[0056] The present application also provides a target positioning device, the target positioning device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the target positioning method as described in any one of the above items; and further comprising a microphone array for obtaining position information of human voice; and at least one lens for obtaining an image.

[0057] The target positioning device provided by the present application adopts the target positioning method in the above embodiment, which can solve the technical problem of low accuracy of identifying speakers by existing tracking cameras. Compared with the prior art, the beneficial effects of the target positioning device provided by the present application are the same as those of the target positioning method provided by the above embodiment, and other technical features in the target positioning device are the same as those disclosed in the above embodiment method, which will not be described in detail here.

[0058] The present application provides a video conferencing device, which includes the target positioning device as described above.

[0059] The video conferencing device provided by the present application adopts the target positioning method in the above embodiment, which can solve the technical problem of low accuracy of speaker recognition by existing tracking cameras. Compared with the prior art, the beneficial effects of the video conferencing device provided by the present application are the same as the beneficial effects of the target positioning method provided by the above embodiment, and other technical features in the video conferencing device are the same as the features disclosed in the method of the previous embodiment, which will not be described in detail here.

[0060] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0061] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

[0062] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the target positioning method in the above-mentioned embodiment.

[0063] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0064] The computer-readable storage medium may be included in the video conferencing device; or may exist independently without being installed in the video conferencing device.

[0065] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned target positioning method, and can solve the technical problem that the existing tracking camera has low accuracy in identifying speakers. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the target positioning method provided in the above-mentioned embodiment, and are not described in detail here.

[0066] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A target positioning method, applied to a target positioning device, characterized in that: The target positioning device includes at least one lens, and the method includes: Get the location information of human voice; adjusting the image acquired by the lens according to the position information; Acquire multiple lip key points of at least one face image of the image, and process the multiple lip key points based on a pre-trained lip movement recognition model to determine the lip state of the face image; wherein the lip state includes a moving state and a static state; When it is determined that the lip state of at least one face image is in a moving state, it is determined that there is at least one speaker in the image, and the camera is controlled to obtain a close-up image of the speaker.

2. The target positioning method according to claim 1, characterized in that: The lip key points include upper inner lip key points and lower inner lip key points, and the training method of the lip movement recognition model includes: Constructing a first one-dimensional feature input according to the distances between a plurality of upper inner lip key points and a plurality of lower inner lip key points; The pre-trained model is trained according to the first one-dimensional feature input to obtain the lip movement recognition model.

3. The target positioning method according to claim 2, characterized in that: The lip key points further include an upper outer lip key point and a lower outer lip key point, and after constructing the first one-dimensional feature input according to the distances between the plurality of upper inner lip key points and the plurality of lower inner lip key points, the method further includes: Constructing a second one-dimensional feature input according to the distances between a plurality of upper outer lip key points and a plurality of lower inner lip key points; and / or, constructing a third one-dimensional feature input according to distances between a plurality of lower outer lip key points and a plurality of upper inner lip key points; And / or, constructing a fourth one-dimensional feature input according to distances between a plurality of upper outer lip key points and a plurality of lower outer lip key points.

4. The target positioning method according to claim 1, characterized in that: The training method of the lip movement recognition model includes: Establishing a coordinate relationship between a plurality of lip key points; Constructing a two-dimensional feature input according to the coordinates of the plurality of lip key points; The pre-trained model is trained according to the two-dimensional feature input to obtain the lip movement recognition model.

5. The target positioning method according to claim 1, characterized in that: The step of obtaining a plurality of lip key points of at least one face image of the image comprises: Processing the image to determine a face image in the image; Processing the facial image to determine a plurality of facial key points of the facial image; Filter out multiple lip key points from multiple face key points.

6. The target positioning method according to claim 1, characterized in that: The lens includes a panoramic lens, and adjusting the image acquired by the lens according to the position information includes: Controlling the panoramic lens to acquire images; Extracting an image corresponding to the position information from the image; The extracted image is processed to obtain a face image in the extracted image.

7. The target positioning method according to claim 6, characterized in that: The lens also includes a close-up shot, and adjusting the image acquired by the lens according to the position information also includes: Determining the distance between the target positioning device and the sound source position of the human voice according to the position of the target positioning device and the position information; When it is determined that the distance between the target positioning device and the sound source of the human voice is greater than a preset distance, adjusting the rotation direction of the close-up lens according to the position information; The magnification of the close-up camera is adjusted so that the close-up camera can recognize a human face image.

8. A target positioning device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the target positioning method as described in any one of claims 1 to 7; and further comprises a microphone array for acquiring position information of human voice; and at least one lens for acquiring images.

9. A video conferencing device, characterized in that: Comprising the target positioning device as claimed in claim 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the target positioning method according to any one of claims 1 to 7 are implemented.

Images