Active focusing method and panoramic image acquisition device

Abstract

An active focusing method is applied to a panoramic image acquisition device. The active focusing method comprises the following steps: receiving a plurality of sound signals in a space from a microphone array; obtaining at least one sound source orientation according to the plurality of sound signals; obtaining at least one image picture close to the sound source orientation; analyzing and comparing the sound signal and the image frame to select one of the plurality of lens modules as a reference lens module, the corresponding field of view of which covers the main speaker; obtaining an automatic focusing value of the reference lens module to the main speaker; the processor adjusts the plurality of lens modules according to the automatic focusing value, so that the plurality of lens modules have the same focal length; and the processor respectively obtains the adjusted image pictures from the plurality of lens modules and splices the adjusted image pictures into a panoramic image picture.

Description

Technical Field

[0001] This invention relates to an active focusing method, and more particularly to an active focusing method applied in a panoramic image acquisition device. Background Technology

[0002] With advancements in optical and image processing technologies, image acquisition devices with multiple lenses have emerged. For example, panoramic image acquisition devices (or 360-degree surround view cameras) use two to four lenses arranged in a ring to capture multiple images with different field-of-view ranges. These images are then stitched together into a single panoramic image using image processing techniques. Additionally, some video conferencing systems utilize panoramic image acquisition devices to obtain a panoramic view of the meeting space.

[0003] However, existing panoramic image acquisition devices can only use multiple fixed-focus lenses, thus preventing focal length adjustment and sometimes resulting in out-of-focus images. If a panoramic image acquisition device uses multiple active focusing lenses, stitching the images together becomes impossible due to the different focal lengths of each lens, necessitating improvement. Summary of the Invention

[0004] In view of the above-mentioned problems, the main objective of the present invention is to provide an active focusing method and a panoramic image acquisition device using the active focusing method. By analyzing and comparing the sound signal and the sound signal of the image, a reference lens module is selected, and the focal length of multiple lens modules is adjusted according to the autofocus value of the reference lens module, so as to solve the problem that known panoramic image acquisition devices are difficult to use active focusing lenses.

[0005] To achieve the above objectives, the present invention provides an active focusing method applied to a panoramic image acquisition device. The panoramic image acquisition device includes multiple lens modules, a microphone array, and a processor. The active focusing method includes the following steps: the processor receives multiple sound signals from the microphone array within a space; the processor obtains the location of at least one sound source based on the multiple sound signals; the processor acquires at least one image frame from multiple lens modules near the location of the at least one sound source, which includes at least one speaker; the processor analyzes and compares the multiple sound signals with the at least one image frame to select one of the multiple lens modules as a reference lens module, the field of view of the reference lens module covering the main speaker; the processor obtains the autofocus value of the reference lens module for the main speaker; the processor adjusts the multiple lens modules based on the autofocus value so that the multiple lens modules have the same focal length; and the processor acquires the adjusted image frames from the multiple lens modules respectively and stitches them together to form a panoramic image frame.

[0006] According to one embodiment of the present invention, the microphone array includes a plurality of microphones, which are spaced apart in the panoramic image acquisition device. The active focusing method further includes the following steps: the processor divides the space into a plurality of directional spaces according to the field of view corresponding to the plurality of lens modules; and the plurality of microphones respectively pick up sound from the plurality of directional spaces to obtain a plurality of sound signals.

[0007] According to an embodiment of the present invention, each sound signal includes directional information. When the average volume of each plurality of sound signals is greater than a threshold, the processor obtains the directional information corresponding to each plurality of sound signals and marks it as at least one sound source directional information.

[0008] According to an embodiment of the present invention, the active focusing method further includes the following steps: the processor analyzes and compares multiple audio signals with at least one image frame to calculate the integral value of at least one speaker; and the processor selects the main speaker based on the highest integral value.

[0009] According to one embodiment of the present invention, when the average volume of multiple sound signals is greater than a threshold, the processor adds a first weight score to obtain an integral value.

[0010] According to one embodiment of the present invention, the processor performs image recognition on facial movements in at least one image frame. When the facial movements match the sound signal, the processor adds a second weight score to obtain an integral value.

[0011] According to one embodiment of the present invention, the processor includes a storage unit that stores a focus parameter table. A reference lens module executes an autofocus program, and the processor obtains the autofocus value based on the focus parameter table.

[0012] According to one embodiment of the present invention, the microphone array includes a plurality of microphones. The plurality of microphones are at least partially located between two adjacent lens modules.

[0013] To achieve the above objectives, the present invention further provides a panoramic image acquisition device, comprising multiple lens modules, a microphone array, and a processor. The processor is electrically connected to the multiple lens modules and the microphone array. The processor performs the following steps according to an active focusing method: receiving multiple sound signals from the microphone array within a space; obtaining the location of at least one sound source based on the multiple sound signals; acquiring at least one image frame from the multiple lens modules near the location of the at least one sound source, which includes at least one speaker; analyzing and comparing the multiple sound signals and the at least one image frame to select one of the multiple lens modules as a reference lens module, the field of view of the reference lens module covering the main speaker; obtaining the autofocus value of the reference lens module for the main speaker; adjusting the multiple lens modules according to the autofocus value so that the multiple lens modules have the same focal length; and acquiring the adjusted image frames from the multiple lens modules respectively, and stitching them together to form a panoramic image frame.

[0014] As described above, the active focusing method and panoramic image acquisition device of the present invention analyze and compare sound signals and image frames to select one of multiple lens modules as a reference lens module, and the field of view corresponding to the reference lens module covers the main speaker. The autofocus value of the reference lens module for the main speaker is obtained, and all lens modules are adjusted according to the autofocus value to ensure they have the same focal length. Finally, each lens module acquires an adjusted image frame with the same focal length, and these images are stitched together to form a panoramic image. Because the adjusted image frames have the same focal length, the inability to stitch images due to different focal lengths is avoided, and the resulting panoramic image will not be distorted or blurred due to stitching. Therefore, the panoramic image acquisition device using the active focusing method of the present invention can use an active focusing lens, i.e., the aforementioned lens modules can be active focusing lens modules. Attached Figure Description

[0015] Figure 1 This is a top view of a panoramic image acquisition device according to an embodiment of the present invention.

[0016] Figure 2A and Figure 2B for Figure 1 The diagram shows the usage environment of the panoramic image acquisition device.

[0017] Figure 3 for Figure 1 The diagram shows a block diagram of a panoramic image acquisition device.

[0018] Figure 4A and Figure 4B This is a flowchart of an active focusing method according to an embodiment of the present invention.

[0019] The attached diagram is described below:

[0020] 1. Panoramic Image Acquisition Device

[0021] Lens modules 11-14

[0022] 20-microphone array

[0023] 21-28 microphones

[0024] 30 processors

[0025] 31 storage units

[0026] P1 Main speaker

[0027] P2 Speaker in a soft voice

[0028] P3 General Staff

[0029] Steps S01 to S10

[0030] S Space

[0031] S1, S2, S3, S4 directional spaces Detailed Implementation

[0032] To better understand the technical content of the present invention, preferred specific embodiments are described below.

[0033] Figure 1 This is a top view of a panoramic image acquisition device according to an embodiment of the present invention. Figure 2A and Figure 2B for Figure 1 The diagram shows the usage environment of the panoramic image acquisition device. Figure 3 for Figure 1 The diagram shows a block diagram of a panoramic image acquisition device. Figure 4A and Figure 4B A flowchart of an active focusing method according to an embodiment of the present invention is provided below. Figure 1 , Figure 2A , Figure 3 , Figure 4A and Figure 4B As shown. First, the panoramic image acquisition device 1 in this embodiment is applied in a video conferencing system to acquire panoramic images of space S (e.g., a conference room space), such as... Figure 2A and Figure 2B As shown. Also, Figure 2A and Figure 2B This diagram illustrates the application of the same panoramic image acquisition device 1 in different sound (speaking) environments. In this embodiment, the panoramic image acquisition device 1 includes multiple lens modules 11-14, a microphone array 20, and a processor 30. The panoramic image acquisition device 1 in this embodiment has four lens modules 11-14, each facing different directions within space S to acquire images with different fields of view. Figure 1 , Figure 2A and Figure 2B As shown, lens module 11 is located on the upper side, lens module 12 on the right side, lens module 13 on the lower side, and lens module 14 on the left side. Lens modules 11-14 are active focusing lens modules, each equipped with an autofocus (AF) module, enabling them to perform autofocus procedures.

[0034] The microphone array 20 includes multiple microphones. In this embodiment, the microphone array 20 has eight microphones 21-28, which are spaced apart on the panoramic image acquisition device 1. Preferably, the microphone array 20 in this embodiment can be an omnidirectional microphone array, with the microphones 21-28 arranged at equal intervals on the panoramic image acquisition device 1, such as... Figure 1 , Figure 2A and Figure 2B As shown. Preferably, in this embodiment, the number of microphones 21-28 is greater than the number of lens modules 11-14, and the ratio is 2:1. Preferably, microphones 21-28 are at least partially located between two adjacent lens modules 11-14. For example, microphones 21, 23, 25, and 27 correspond to lens modules 11, 12, 13, and 14, respectively, while microphones 22, 24, 26, and 28 are located between lens modules 11, 12, 13, and 14, respectively. Specifically, microphone 22 is located between two adjacent lens modules 11 and 12; microphone 24 is located between two adjacent lens modules 12 and 13; microphone 26 is located between two adjacent lens modules 13 and 14; and microphone 28 is located between lens modules 11 and 14.

[0035] like Figure 3 As shown, the processor 30 is electrically connected to the lens modules 11-14 and the microphone array 20 to receive and transmit related parameters, image signals, sound signals, or commands. The processor 30 may be, for example, but not limited to, a central processing unit (CPU) or a microprocessor. In this embodiment, the processor 30 also includes a storage unit 31, which stores data such as... Figure 4A and Figure 4B The active focusing method shown is executed by processor 30 to achieve the function of the present invention. Furthermore, this embodiment only illustrates a preferred embodiment of the present invention; to avoid redundancy, not all possible variations and combinations are described in detail. However, those skilled in the art should understand that not all of the above modules or elements are necessarily necessary. Other known modules or elements with more detail may also be included for implementing the present invention. Each module or element may be omitted or modified as needed, and there may be other modules or elements between any two modules. The operation of each module and element is further explained below according to the steps of the active focusing method. It should be noted that the following step numbers are arranged sequentially for ease of explanation only and are not intended to limit the order of the steps. Multiple steps can be interchanged or performed simultaneously; the present invention is not limited in this respect.

[0036] Step S01: Divide space S into multiple directional spaces S1, S2, S3, and S4 according to the field of view corresponding to lens modules 11 to 14.

[0037] As mentioned above, in this embodiment, the four lens modules 11, 12, 13, and 14 face different directions within space S to acquire images with different field of view ranges. In this embodiment, the processor 30 may preset four directional spaces S1, S2, S3, and S4. Wherein, directional spaces S1, S2, S3, and S4 are the field of view ranges corresponding to lens modules 11, 12, 13, and 14, respectively. Figure 2A and Figure 2B As shown.

[0038] Furthermore, the parameters or image signals transmitted by lens modules 11-14 each have different device information, enabling processor 30 to determine which orientation space S1, S2, S3, or S4 the image is from. For example, when processor 30 receives the image signal from lens module 11, it can determine from the device information that it is an image from orientation space S1.

[0039] It should also be noted that, Figure 2A and Figure 2B The orientation spaces S1, S2, S3, and S4 shown are examples to facilitate the description of the active focusing method of the present invention. The processor 30 can directly determine its relative relationship with the microphones 21 to 28 based on the parameters transmitted by the lens modules 11 to 14 or the device information carried in the image signal, so as to perform subsequent analysis and comparison.

[0040] Step S02: Receive multiple sound signals from space S from microphone array 20.

[0041] In this embodiment, the microphone array 20 has eight microphones 21 to 28, each capable of picking up sound signals from space S. Therefore, the microphone array 20 can acquire up to eight sets of sound signals at the same time. Preferably, the microphones 21 to 28 are spaced at the same distance, so that the eight microphones 21 to 28 can pick up sound from the directional spaces S1, S2, S3, and S4 on an even basis, thereby acquiring multiple sound signals. After receiving multiple sound signals from space S from the microphone array 20, the processor 30 can determine the location of the sound source by the average volume (e.g., amplitude) of the sound signals from different microphones 21 to 28, for example, in step S03.

[0042] Step S03: Obtain the location of at least one sound source based on multiple sound signals.

[0043] In this embodiment, the processor 30 analyzes the average volume of each sound signal from different microphones 21-28 to calculate the location of the sound source, referred to as the sound source location in this embodiment. Specifically, the processor 30's storage unit 31 also stores the relative positions of the four lens modules 11-14 and the eight microphones 21-28. Similarly, the sound signals transmitted by the eight microphones 21-28 each have different device information, referred to as location information in this embodiment. In other words, each sound signal in this embodiment includes location information. Therefore, the processor 30 can determine which directional space S1, S2, S3, S4 the microphones 21-28 transmitting the sound signal are close to through the location information contained in the sound signal.

[0044] Furthermore, the processor 30 obtains the location of at least one sound source based on multiple sound signals. Specifically, there may be multiple sound sources in space S, allowing the processor 30 to obtain multiple sound source locations. In this embodiment, the processor 30 can obtain the sound source location based on the average volume (i.e., amplitude) of each sound signal. Figure 2A The following is an example of the sound (speaking) environment shown. Figure 2A The main speaker P1 (i.e., the speaker in the meeting) is located in directional space S1, while two other speakers P2 are quietly chatting in directional space S2. Microphones 21, 22, 23, 24, and 25, located near the main speaker P1 and the quiet speakers P2, can all receive sound and generate sound signals. Since microphone 22 is closer to the main speaker P1 and microphone 23 is closer to the quiet speakers P2, the processor 30 determines, based on the average volume (i.e., amplitude) of each sound signal and the directional information, that the average volume of the sound signals obtained by microphones 22 and 23 is larger, while the average volume of the sound signals obtained by microphones 21 and 24 is smaller. Therefore, it can be determined that one sound source is located in directional space S1 and near microphone 22, and the other sound source is located in directional space S2 and near microphone 23.

[0045] by Figure 2B The following is an example of the sound (speaking) environment shown. Figure 2B The main speaker P1 (the person speaking in the meeting) and two quiet speakers P2 are both located in directional space S2. Microphones 22, 23, and 24, which are close to the main speaker P1 and the quiet speakers P2, can receive sound and generate sound signals. Since microphone 23 is closest to the main speaker P1 and the quiet speakers P2, the processor 30 determines, based on the average volume (i.e., amplitude) of each sound signal and the directional information, that the average volume of the sound signal obtained by microphone 23 is larger, while the average volume of the sound signals obtained by microphones 22 and 24 is smaller. Therefore, it can be determined that the sound source is located in directional space S2 and close to microphone 23.

[0046] It should be noted that if the sound source is located between two adjacent microphones (e.g., microphones 21 and 22), the location of the sound source can be determined based on the volume and angle of the sound signals obtained by the two microphones 21 and 22.

[0047] Step S04: Obtain at least one image from multiple lens modules 11-14 located near at least one sound source, which includes at least one speaker.

[0048] Next, the processor 30 acquires at least one image from the lens modules 11-14 located near the sound source. Figure 2A For example, the sound sources are located in directional space S1 and near microphone 22, and in directional space S2 and near microphone 23, respectively, and are adjacent to lens modules 11 and 12. Therefore, in step S04, processor 30 obtains two image frames from lens modules 11 and 12, one for directional space S1 and the other for directional space S2. Since processor 30 obtains the image frames based on the sound source location, the image frames contain at least one speaker.

[0049] by Figure 2B For example, the sound source is located in azimuth space S2 and is close to microphone 23, which is adjacent to lens module 12. Therefore, in step S04, processor 30 obtains an image of azimuth space S2 from lens module 12. Since processor 30 obtains the image based on the sound source location, the image contains at least one speaker.

[0050] It should be noted that after the panoramic image acquisition device 1 is powered on, the processor 30 continuously receives image frames from the lens modules 11-14 in the azimuth spaces S1, S2, S3, and S4, and continuously receives sound signals from the microphones 21-28 in space S. If there is a speaker in space S (e.g., a main speaker P1 or a soft-spoken speaker P2), an image frame containing at least one speaker can be obtained in step S04. In other words, step S04 defines obtaining an image frame containing at least one speaker, but in practical applications, the processor 30 continuously receives image frames from the azimuth spaces S1, S2, S3, and S4, as well as sound signals from space S.

[0051] Step S05: Analyze and compare multiple audio signals with at least one video image to calculate the speaker's integral value.

[0052] In this embodiment, the processor 30 can analyze all the image frames to obtain the portraits of each person in each image frame. Figure 2AFor example, the image frame in orientation space S1 contains five figures. Furthermore, the processor 30 can perform image recognition on the facial movements of the figures and can easily determine that there is one speaker (i.e., the main speaker P1) and four ordinary people P3 without mouth movements. Similarly, the image frame in orientation space S2 also contains five figures, but one of them (i.e., the main speaker P1) only has a body part, and their facial movements cannot be identified. The processor 30 can easily determine that the image frame in orientation space S2 contains two speakers (i.e., soft-spoken speakers P2) and two ordinary people P3 without mouth movements. The image frame in orientation space S3 contains three figures, all of whom are ordinary people P3 without mouth movements. The image frame in orientation space S4 contains four figures, all of whom are ordinary people P3 without mouth movements.

[0053] In this embodiment, each speaker (i.e., those identified as having mouth movements) can be assigned an initial score, and weighted scoring conditions can be set. Therefore, a primary speaker P1 in the directional space S1 image and two soft-spoken speakers P2 in the directional space S2 image each have an initial score. Regarding the weighted scoring conditions, the processor 30 can first analyze the sound signal, assess whether the average volume of the sound signal is greater than a threshold, and the processor 30 can compare the image and the sound signal to determine whether the speaker's facial movements in the image match the sound signal. When the average volume of the sound signal is greater than the threshold, the processor 30 can increase the first weighted score. Furthermore, the processor 30 performs image recognition on the facial movements in the image. When the facial movements match the sound signal, the processor 30 can also increase the second weighted score. In this embodiment, the sum of the initial score, the first weighted score (if any), and the second weighted score (if any) is used as the speaker's integral value.

[0054] Specifically, the processor 30 can determine, based on the configuration of the lens modules 11-14 and the microphone array 20, whether the sound signals generated by microphones 21, 22, and 23 contain the voice of the speaker (i.e., the main speaker P1) in the directional space S1, and determine whether the average volume of the sound signals from microphones 21, 22, and 23 is greater than a threshold. For example, if microphone 22 is closest to the speaker (i.e., the main speaker P1) in the directional space S1, the average volume of the sound signal from microphone 22 is greater than the threshold, and a first weight score can be added. However, if the average volume of the sound signals from microphones 21 and 23 is less than the threshold, no first weight score is added. Next, the processor 30 compares the facial movements of the speaker (i.e., the main speaker P1) in the image frame of the directional space S1 with the sound signal from the microphone 22, which has the highest volume. For example, if the timing of the sound signal from microphone 22 matches the mouth movements of the speaker (i.e., the main speaker P1) in the image, a second weight score can be added. Figure 2AFor example, the integral value of the speaker (i.e. the main speaker P1) in the image of the azimuth space S1 is the sum of the original score, the first weight score, and the second weight score.

[0055] Similarly, the processor 30 can determine, based on the configuration of the lens modules 11-14 and the microphone array 20, whether the sound signals generated by microphones 22, 23, and 24 contain the voice of the speaker (i.e., the soft-spoken speaker P2) in the directional space S2, and then determine whether the average volume of the sound signals from microphones 22, 23, and 24 is greater than a threshold. Since the speaker in directional space S2 is the soft-spoken speaker P2 engaging in quiet conversation, the average volume of the sound signals from microphones 22, 23, and 24 is still less than the threshold, and the first weight score cannot be increased. Similarly, the processor 30 compares the facial movements of the speaker (i.e., the soft-spoken speaker P2) in the image frame of directional space S2 with the sound signal from the microphone 22, which has the highest volume. Since the sound signal from microphone 22 does not match the facial movements of the speaker (i.e., the soft-spoken speaker P2) in the image frame of directional space S2, the second weight score cannot be increased. Figure 2A For example, the integral value of the speaker (i.e., the soft-spoken speaker P2) in the image of the azimuth space S2 is the original score.

[0056] by Figure 2B For example, processor 30 can analyze the image frames in directional spaces S1, S2, S3, and S4, and perform simple image recognition on the facial movements of the people in the image frames. Specifically, the image frame in directional space S1 contains four ordinary people (P3) without mouth movements. The image frame in directional space S2 contains three speakers (i.e., the main speaker P1 and the soft-spoken speaker P2) and two ordinary people (P3) without mouth movements. The image frame in directional space S3 contains three ordinary people (P3) without mouth movements. The image frame in directional space S4 contains four ordinary people (P3) without mouth movements.

[0057] Similarly, each speaker (i.e., those identified with mouth movements) is assigned an initial score, and weighted scoring conditions are set. Therefore, the three speakers in the directional space S2 image (i.e., the main speaker P1 and the soft-spoken speaker P2) all have initial scores. Next, the processor 30 analyzes the sound signals to evaluate whether to increase the first weight score and the second weight score (i.e., the weighted scoring conditions). Specifically, the processor 30 can determine from the configuration of the lens modules 11-14 and the microphone array 20 that the sound signals generated by microphones 22, 23, and 24 contain the voices of the speakers in directional space S2 (i.e., the main speaker P1 and the soft-spoken speaker P2), and determine whether the average volume of the sound signals from microphones 22, 23, and 24 is greater than a threshold. For example, if microphone 23 is closest to the speaker in directional space S2 (i.e., the soft-spoken speaker P2 and the main speaker P1), the average volume of the sound signal from microphone 23 is greater than the threshold, and the first weight score can be increased. However, the average volume of the sound signals from microphones 22 and 24 is less than the threshold, and the first weight score is not increased.

[0058] Next, the processor 30 compares the facial movements of the speakers (i.e., the main speaker P1 and the soft-spoken speaker P2) in the image frame of the directional space S2 with the sound signal from the microphone 23 with the loudest volume. Figure 2B For example, the timing of the sound signal from microphone 23 corresponds to the mouth movements of a speaker behind the camera (i.e., the main speaker P1) in the image, but not to the mouth movements of the two speakers in front (i.e., the soft-spoken speakers P2). Therefore, the speaker behind the camera (i.e., the main speaker P1) can gain a second weighted score; while the speaker in front (i.e., the soft-spoken speakers P2) cannot gain a second weighted score. Figure 2B For example, the integral value of a speaker behind (the main speaker P1) in the image of the directional space S2 is the sum of the original score, the first weighted score, and the second weighted score, while the integral value of the two speakers in front (i.e., the soft-spoken speaker P2) is the sum of the original score and the first weighted score.

[0059] Preferably, regarding weighted scoring, the processor 30 in this embodiment first determines the average volume of the sound signal and then compares the image and the sound signal. In other embodiments, the processor 30 may also first compare the image and the sound signal and then determine the average volume of the sound signal; this invention is not limited in this respect. In other words, this invention does not limit the order in which the first weighted score and the second weighted score are calculated, and they can also be performed simultaneously.

[0060] Step S06: Select the main speaker based on the highest score.

[0061] The processor 30 selects the highest score from the scores of each speaker calculated in step S05; in this embodiment, this is referred to as the highest score. Figure 2A For example, the speaker's score in the image frame of azimuth space S1 is the sum of the original score, the first weighted score, and the second weighted score, which is the highest score. In this embodiment, the speaker corresponding to the highest score is also determined by the processor 30 to be the speaker who is giving an opinion in the meeting, i.e., the main speaker P1.

[0062] by Figure 2B For example, the score of a speaker in the image frame of azimuth space S2 is the sum of the original score, the first weight score and the second weight score, which is the highest score, and is determined by processor 30 to be the main speaker P1.

[0063] Step S07: Select one of the multiple lens modules 11 to 14 as a reference lens module. The field of view of the reference lens module covers the main speaker.

[0064] Next, with Figure 2A For example, the processor 30 selects lens module 11 with a field of view of azimuth space S1 from multiple lens modules 11-14 as the reference lens module. Figure 2B For example, the processor 30 selects lens module 12, whose field of view is in the azimuth space S2, from among multiple lens modules 11 to 14 as the reference lens module. Therefore, the field of view corresponding to the reference lens module (i.e., lens module 11) covers the main speaker P1 (i.e., the speaker corresponding to the highest integral value).

[0065] It should be noted that, Figure 2A and Figure 2B In the example, only the field of view of lens module 11 or lens module 12 covers the entirety of the main speaker P1, so lens module 11 or lens module 12 is directly selected. In other embodiments, if the field of view of multiple lens modules (e.g., lens modules 11 and 12) covers the entirety of the main speaker P1, the distance between lens modules 11 and 12 and the main speaker P1 can be further analyzed, and the lens module closest to the main speaker P1 (i.e., one of lens modules 11 and 12) can be selected.

[0066] Step S08: Obtain the autofocus value of the reference lens module for the main speaker.

[0067] As mentioned above, lens modules 11 to 14 in this embodiment are all active focusing lens modules. Therefore, the reference lens module (i.e., lens module 11 or lens module 12) can perform an autofocus program on the main speaker P1 (i.e. the speaker corresponding to the highest score) to obtain a focus value and use it as the autofocus value in this embodiment.

[0068] Specifically, in this embodiment, the storage unit 31 of the processor 30 stores a focus parameter table. The focus parameter table includes multiple focus distances and their corresponding focus values. Furthermore, in this embodiment, lens modules 11-14 each include a focus motor (not shown). The focus distance is the distance from lens modules 11-14 to the object being focused on, and the focus value can be the number of drive steps of the focus motor.

[0069] After the panoramic image acquisition device 1 of this embodiment is powered on, the lens modules 11-14 acquire image images with different field of view ranges, and simultaneously acquire the focusing distance between each lens module 11-14 and each person in space S. Taking lens module 11 as an example, the focusing distance between lens module 11 and the main speaker P1 and four ordinary persons P3 in the directional space S1 can be acquired respectively. In step S06, the main speaker P1 is identified based on the highest integral value. In step S07, a reference lens module (i.e., lens module 11 or lens module 12) is selected. Next, in step S08, the reference lens module (i.e., lens module 11 or lens module 12) performs an autofocus program on the main speaker P1, and the processor 30 obtains the focusing value of the main speaker P1 (e.g., the number of drive steps of the focusing motor) according to the focusing parameter table, and defines it as the autofocus value.

[0070] Step S09: Adjust multiple lens modules 11 to 14 according to the autofocus value so that the multiple lens modules 11 to 14 have the same focal length.

[0071] The processor 30 adjusts the focal length of all lens modules 11 to 14 according to the autofocus value obtained in step S07, so that the lens modules 11 to 14 respectively obtain image images with the same focal length.

[0072] Step S10: Obtain the adjusted image images from multiple lens modules 11 to 14 respectively, and stitch them together to form a panoramic image image.

[0073] The processor 30 acquires adjusted image images from lens modules 11-14, that is, image images adjusted to the same focal length. Finally, the processor 30 stitches the adjusted image images acquired from lens modules 11-14 into a panoramic image. Since the adjusted image images have the same focal length, the situation where images cannot be stitched due to different focal lengths can be avoided, and the resulting panoramic image will not be distorted or blurry due to stitching.

[0074] In summary, the active focusing method and panoramic image acquisition device of the present invention, by analyzing and comparing sound signals and image frames, selects one of the multiple lens modules of the panoramic image acquisition device as a reference lens module, and the field of view corresponding to the reference lens module covers the main speaker. The autofocus value of the reference lens module for the main speaker is obtained, and all lens modules are adjusted according to the autofocus value so that all lens modules have the same focal length. Finally, the lens modules respectively acquire adjusted image frames with the same focal length and stitch them together to form a panoramic image. Because the adjusted image frames have the same focal length, the situation where images cannot be stitched due to different focal lengths can be avoided, and the resulting panoramic image will not be distorted or blurry due to stitching. Therefore, the panoramic image acquisition device using the active focusing method of the present invention can use an active focusing lens, that is, the aforementioned lens modules can be active focusing lens modules.

[0075] It should be noted that the above embodiments are examples for illustrative purposes only, and the scope of the claims of this invention should be determined by the scope defined in the claims, and not limited to the above embodiments.

Claims

1. An active focusing method, characterized in that, An active focusing method for a panoramic image acquisition device, comprising multiple lens modules, a microphone array, and a processor, includes the following steps: The processor receives multiple sound signals from the microphone array within the space; The processor obtains the location of at least one sound source based on the plurality of sound signals; The processor acquires at least one image from the plurality of lens modules located near the at least one sound source, the at least one image containing at least one speaker; The processor analyzes and compares the plurality of sound signals with the at least one image frame to select one of the plurality of lens modules as a reference lens module, wherein the field of view of the reference lens module covers the main speaker; The processor obtains the autofocus value of the reference lens module for the main speaker; The processor adjusts the plurality of lens modules according to the autofocus value, so that the plurality of lens modules have the same focal length; and The processor obtains the adjusted image images from the multiple lens modules respectively, and stitches them together to form a panoramic image.

2. The active focusing method as described in claim 1, characterized in that, The microphone array includes multiple microphones spaced apart in the panoramic image acquisition device, and the active focusing method further includes the following steps: The processor divides the space into multiple directional spaces based on the field of view corresponding to the multiple lens modules; and The plurality of microphones respectively pick up sound from the plurality of directional spaces to obtain the plurality of sound signals.

3. The active focusing method as described in claim 1, characterized in that, Each of the plurality of sound signals includes directional information, and the processor obtains the directional information of the at least one sound source based on the average volume of each of the plurality of sound signals and the directional information.

4. The active focusing method as described in claim 1, characterized in that, It also includes the following steps: The processor analyzes and compares the plurality of audio signals with the at least one image frame to calculate the integral value of the at least one speaker; and The processor selects the primary speaker based on the highest score.

5. The active focusing method as described in claim 4, characterized in that, When the average volume of the multiple sound signals is greater than a threshold, the processor adds a first weight score to obtain an integral value.

6. The active focusing method as described in claim 4, characterized in that, The processor performs image recognition on facial movements in at least one image frame. When the facial movements match the sound signal, the processor adds a second weight score to obtain an integral value.

7. The active focusing method as described in claim 1, characterized in that, The processor includes a storage unit that stores a focus parameter table. The reference lens module executes an autofocus program, and the processor obtains the autofocus value based on the focus parameter table.

8. The active focusing method as described in claim 1, characterized in that, The microphone array includes multiple microphones, which are at least partially located between two adjacent lens modules.

9. A panoramic image acquisition device, characterized in that, include: Multiple lens modules; Microphone array; as well as The processor, electrically connected to the plurality of lens modules and the microphone array, performs the following steps according to the active focusing method: Multiple sound signals are received from the space by the microphone array; At least one sound source location is obtained based on the multiple sound signals; Acquire at least one image from the plurality of lens modules located near the at least one sound source, the at least one image containing at least one speaker; The plurality of audio signals are analyzed and compared with the at least one video image to select one of the plurality of camera modules as a reference camera module, wherein the field of view of the reference camera module covers the main speaker; Obtain the autofocus value of the reference lens module for the main speaker; Adjust the plurality of lens modules according to the autofocus value so that the plurality of lens modules have the same focal length; and The adjusted images are obtained from the multiple lens modules and stitched together to form a panoramic image.

Images