Pan-tilt fusion control method, device and equipment integrating acoustic positioning and medium

Abstract

The invention relates to the technical field of intelligent robots, and discloses a cradle head fusion control method, device and equipment integrating acoustic localization and a medium, and the method comprises the steps: collecting target sound source information through an acoustic recognition module, and transmitting the target sound source information to a core control module through a communication module; the target sound source information is analyzed through the core control module, a target pose is determined, and the visual identification module is controlled to the target pose through the motor driving module; and collecting target visual information through a visual identification module, and fusing the target visual information with the target sound source information to generate fusion control information. Through the above mode, the sound source information is collected through the acoustic recognition module, the system is not limited by the visual field, the sound source can be searched, and the target searching range is expanded. Acoustic positioning is utilized to assist visual identification, the dependence on a complex visual algorithm is reduced, the positioning difficulty of the visual algorithm is simplified, and the positioning accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent robot technology, and in particular to a pan-tilt fusion control method, device, equipment and medium integrated with acoustic positioning. Background Art

[0002] In traditional pan-tilt technology, visual guidance is the main means of controlling motors and collecting target point information. This method relies on the field of view of the visual camera to search for targets. Once the target is beyond the field of view, information collection will fail. In addition, relying solely on visual recognition for positioning requires complex visual algorithms, which increases the complexity and computational burden of the system. Existing pan-tilt technology usually uses a two-dimensional pan-tilt driven by a servo. After visual recognition by the main control unit, the pan-tilt is controlled through serial communication. Although this solution is rich in information, it has problems with positioning difficulty and delay.

[0003] In order to solve the above problems, some technical solutions have proposed a composite audio-visual perception and positioning system using an acoustic array and a multispectral camera. This system combines an acoustic array component with a multispectral camera device to overcome the limitations of the acoustic array in three-dimensional spatial positioning and the limitations of binocular recognition in complex environments. Through the combination of an acoustic array component and a multispectral camera device, the system achieves the coordination of acoustic detection and binocular recognition, improves the accuracy of recognition and positioning, and improves adaptability to complex environments.

[0004] Existing PTZ technology has certain limitations in target search and positioning, especially in terms of visual field limitations and complex visual algorithm requirements. Therefore, how to improve the accuracy of PTZ positioning by combining acoustics and vision has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides a pan-tilt fusion control method, device, equipment and medium integrating acoustic positioning, which improves the accuracy of pan-tilt positioning by combining acoustics and vision.

[0006] In a first aspect, the present application provides a pan-tilt fusion control method integrating acoustic positioning, the method comprising:

[0007] Collecting target sound source information through the acoustic recognition module, and transmitting the target sound source information to the core control module through the communication module;

[0008] Parsing the target sound source information through the core control module to determine the target posture, and controlling the visual recognition module to the target posture through the motor drive module;

[0009] The target visual information is collected through the visual recognition module, and the target visual information is fused with the target sound source information to generate fusion control information.

[0010] In a second aspect, the present application also provides a pan-tilt fusion control device with integrated acoustic positioning, the device comprising:

[0011] a target sound source information acquisition module, used to acquire target sound source information through the acoustic recognition module, and transmit the target sound source information to the core control module through the communication module;

[0012] A target posture adjustment module, used for parsing the target sound source information through the core control module, determining the target posture, and controlling the visual recognition module to the target posture through the motor drive module;

[0013] The fusion control information generating module is used to collect the target visual information through the visual recognition module, and fuse the target visual information with the target sound source information to generate fusion control information.

[0014] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the pan-tilt fusion control method with integrated acoustic positioning as described above when executing the computer program.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the pan-tilt fusion control method with integrated acoustic positioning as described above.

[0016] The present application discloses a pan-tilt fusion control method, device, equipment and medium with integrated acoustic positioning. The pan-tilt fusion control method with integrated acoustic positioning includes collecting target sound source information through the acoustic recognition module, and transmitting the target sound source information to the core control module through the communication module; parsing the target sound source information through the core control module, determining the target posture, and controlling the visual recognition module to the target posture through the motor drive module; collecting target visual information through the visual recognition module, and fusing the target visual information with the target sound source information to generate fusion control information. In the above manner, the present application collects sound source information through the acoustic recognition module, is not limited by the visual field, can search for the sound source, and expands the scope of target search. Using acoustic positioning to assist visual recognition reduces the dependence on complex visual algorithms, simplifies the positioning difficulty of visual algorithms, and improves the accuracy of positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a schematic flow chart of a pan-tilt fusion control method with integrated acoustic positioning provided in the first embodiment of the present application;

[0019] Figure 2 A schematic diagram of the structure of a pan / tilt fusion control system with integrated acoustic positioning provided in the first embodiment of the present application;

[0020] Figure 3 A schematic diagram of the operation of the pan / tilt fusion control system with integrated acoustic positioning provided in the first embodiment of the present application;

[0021] Figure 4 is a schematic flow chart of a pan-tilt fusion control method with integrated acoustic positioning provided in the second embodiment of the present application;

[0022] Figure 5 A schematic diagram of acoustic positioning of a pan / tilt fusion control system with integrated acoustic positioning provided in the second embodiment of the present application;

[0023] Figure 6 A schematic block diagram of a pan / tilt fusion control device with integrated acoustic positioning provided in an embodiment of the present application;

[0024] Figure 7 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0027] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0028] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0029] The embodiments of the present application provide a pan-tilt fusion control method, device, equipment and medium integrated with acoustic positioning. Among them, the pan-tilt fusion control method integrated with acoustic positioning can be applied to a server, and the sound source information is collected through an acoustic recognition module, which is not limited by the visual field and can search for the sound source, thereby expanding the scope of target search. The use of acoustic positioning to assist visual recognition reduces the dependence on complex visual algorithms, simplifies the positioning difficulty of the visual algorithm, and improves the accuracy of positioning. Among them, the server can be an independent server or a server cluster.

[0030] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0031] See also Figure 1 , Figure 1 This is a schematic flow chart of a pan-tilt fusion control method with integrated acoustic positioning provided in the first embodiment of the present application. The pan-tilt fusion control method with integrated acoustic positioning can be applied to a pan-tilt fusion control system with integrated acoustic positioning, which is used to collect sound source information through an acoustic recognition module, is not limited by the visual field, can search for the sound source, and expands the scope of target search. Using acoustic positioning to assist visual recognition reduces the dependence on complex visual algorithms, simplifies the positioning difficulty of the visual algorithm, and improves the accuracy of positioning.

[0032] like Figure 1 As shown, the pan-tilt fusion control method integrating acoustic positioning specifically includes steps S10 to S30.

[0033] Step S10, collecting target sound source information through the acoustic recognition module, and transmitting the target sound source information to the core control module through the communication module;

[0034] Step S20: parsing the target sound source information through the core control module, determining the target posture, and controlling the visual recognition module to the target posture through the motor drive module;

[0035] Step S30 collects target visual information through the visual recognition module, and fuses the target visual information with the target sound source information to generate fusion control information.

[0036] In one embodiment, the pan-tilt fusion control method with integrated acoustic positioning is applied to the pan-tilt fusion control system with integrated acoustic positioning. The pan-tilt fusion control system with integrated acoustic positioning includes an acoustic recognition module, a communication module, a core control module, a motor drive module and a visual recognition module. In addition, the pan-tilt fusion control system with integrated acoustic positioning also includes a power supply module. The acoustic system includes a sound source recognition module, which is mainly responsible for the collection of sound sources and the recognition function of specific sound source types. The core control module determines the position and posture of the identified specific sound source and outputs it to the motor. The motor drive system includes a motor drive module, whose main function is to receive the posture decomposition information of the core control module to control the motor and finally locate the target detection point. The visual system includes a visual recognition module, which is composed of visible light and infrared cameras and is mainly responsible for the collection of image information. The collected visual information is given to the core control module for information analysis and fusion, and finally output or storage. The communication system is responsible for information exchange between systems, and the power supply system is responsible for power supply.

[0037] like Figure 2 As shown, Figure 2 A schematic structural diagram of a pan-tilt fusion control system with integrated acoustic positioning provided for the first embodiment of the present application, wherein the acoustic system is composed of at least one acoustic recognition module, and the visual system is composed of at least one visual recognition module. Figure 3 The schematic diagram of the operation of the pan-tilt fusion control system with integrated acoustic positioning provided in the first embodiment of the present application. Figure 2 It can be seen that the rotation axis 1 connects the acoustic system and the pan / tilt, and the rotation axis 2 connects the visual system and the pan / tilt. By rotating the rotation axis 1, the position of the acoustic system is adjusted to capture the external target sound source information; by rotating the rotation axis 2, the position of the visual system is adjusted to cooperate with the acoustic system to collect the target visual information.

[0038] Specifically, an acoustic recognition module is started, which includes a microphone or other acoustic sensor. The acoustic recognition module monitors the sounds in the environment in real time, identifies and collects characteristic information of the target sound source, such as frequency, waveform, intensity, etc.

[0039] The collected target sound source information is transmitted to the core control module through the communication module (which may be a wireless or wired network) to ensure the accuracy and real-time performance of data transmission. After receiving the sound source information, the core control module performs analysis and processing, and uses the sound source localization algorithm to determine the position and direction (target posture) of the sound source.

[0040] The core control module sends the analyzed target posture information to the motor drive module. The motor drive module controls the visual recognition module (such as a camera) to move to the target posture. After the visual recognition module reaches the target posture, it starts to collect the target's visual information, which includes color, shape, texture, motion and other features.

[0041] The collected target visual information is fused with the previously collected target sound source information, and a specific algorithm (such as a sensor fusion algorithm) is used to generate fusion control information to improve the accuracy and robustness of recognition. The fusion control information may include the target's position, motion state, recognition results, etc. This information is used for further decision-making, control or feedback to the user.

[0042] The present embodiment discloses a pan-tilt fusion control method, device, equipment and medium with integrated acoustic positioning. The pan-tilt fusion control method with integrated acoustic positioning includes collecting target sound source information through the acoustic recognition module, and transmitting the target sound source information to the core control module through the communication module; parsing the target sound source information through the core control module, determining the target posture, and controlling the visual recognition module to the target posture through the motor drive module; collecting target visual information through the visual recognition module, and fusing the target visual information with the target sound source information to generate fusion control information. In the above manner, the present application collects sound source information through the acoustic recognition module, is not limited by the visual field, can search for the sound source, and expands the scope of target search. Using acoustic positioning to assist visual recognition reduces the dependence on complex visual algorithms, simplifies the positioning difficulty of visual algorithms, and improves the accuracy of positioning.

[0043] See also Figure 4 , Figure 4 This is a schematic flow chart of a pan-tilt fusion control method with integrated acoustic positioning provided in the second embodiment of the present application. The pan-tilt fusion control method with integrated acoustic positioning can be applied to a pan-tilt fusion control system with integrated acoustic positioning, which is used to collect sound source information through an acoustic recognition module, is not limited by the visual field, can search for the sound source, and expands the scope of target search. Using acoustic positioning to assist visual recognition reduces the dependence on complex visual algorithms, simplifies the positioning difficulty of the visual algorithm, and improves the accuracy of positioning.

[0044] based on Figure 1 The embodiment shown, this embodiment Figure 4 As shown, step S20 includes steps S11 to S13.

[0045] Step S11, obtaining historical sound source sample information;

[0046] Step S12: parsing the historical sound source sample information through the core control module to extract the historical sound source features and / or historical sound source frequencies of the historical sound source sample information;

[0047] Step S13: determining the sound source type of the historical sound source sample information according to each of the historical sound source features and / or the historical sound source frequency, and determining the sample pose of the historical sound source sample information according to the sound source type of the historical sound source sample information.

[0048] like Figure 5 As shown, Figure 5 A schematic diagram of acoustic positioning of a pan-tilt fusion control system with integrated acoustic positioning provided in the second embodiment of the present application. The specific sound source emitted by the detection target will be acoustically guided by the pan-tilt, and after positioning is completed, visual information is collected and analyzed and fused.

[0049] In one embodiment, the specific sound source emitted by the detection target will be acoustically guided by the pan-tilt head, and after positioning is completed, visual information is collected and analyzed and fused. At the same time, the pan-tilt head can perform deep learning on the collected information, which can help train the pan-tilt head's sound source recognition and sound source positioning. Through deep learning of a large amount of sample sound source data collected, the set specific sound source type can be quickly tracked to achieve rapid positioning.

[0050] Collect historical sound source sample data, which may include audio files of various sound sources recorded in the past, use the core control module to parse the historical sound source sample information, use specific algorithms to analyze the audio files, and extract key features.

[0051] The historical sound source features and / or historical sound source frequencies in the historical sound source sample information are extracted by audio signal processing technology, such as extracting MFCC (Mel Frequency Cepstrum Coefficient) features.

[0052] Based on the extracted historical sound source features and / or historical sound source frequencies, a classification algorithm is used to determine the type of sound source. Specifically, a machine learning model, such as a support vector machine (SVM) or a deep learning network, may be involved to classify the sound source. Once the type of sound source is determined, the sample pose can be determined based on the type of sound source, and a sound source localization algorithm, such as a Matched field processing (MFP) algorithm, can be used to determine the location and direction of the sound source.

[0053] In some cases, the sound can be simulated by a physical model to generate more data to extract features and train the model, so as to more accurately determine the sample pose. All the above feature extraction steps are integrated into one function to extract all necessary features from the audio file to form a feature vector for subsequent machine learning model training and classification.

[0054] The present embodiment discloses a pan-tilt fusion control method, device, equipment and medium with integrated acoustic positioning. The pan-tilt fusion control method with integrated acoustic positioning includes collecting target sound source information through the acoustic recognition module, and transmitting the target sound source information to the core control module through the communication module; obtaining historical sound source sample information; parsing the historical sound source sample information through the core control module, extracting historical sound source features and / or historical sound source frequencies of the historical sound source sample information; determining the sound source type of the historical sound source sample information according to each of the historical sound source features and / or the historical sound source frequencies, and determining the sample pose of the historical sound source sample information according to the sound source type of the historical sound source sample information; parsing the target sound source information through the core control module, determining the target pose, and controlling the visual recognition module to the target pose through the motor drive module; collecting target visual information through the visual recognition module, and fusing the target visual information with the target sound source information to generate fusion control information. Through the above method, the present application collects the target sound source information through the acoustic recognition module, and combines the historical sound source sample information to more accurately determine the position and type of the sound source, thereby improving the positioning accuracy of the target posture, and realizes multimodal recognition of the target by fusing the sound source information and visual information, thereby enhancing the recognition ability of the target. Especially when the visual information is limited, the sound source information can be used as a supplement to quickly collect the sound source information and quickly process it through the core control module to realize rapid positioning and recognition of the target, thereby improving the accuracy of positioning.

[0055] In a more preferred embodiment, step S20 includes:

[0056] Extracting target sound source features of the target sound source information through the core control module;

[0057] Determining a target sound source type of the target sound source information according to the target sound source feature and the historical sound source feature;

[0058] The sample pose corresponding to the sound source type matching the target sound source type in the historical sound source sample information is determined as the target pose.

[0059] Specifically, the acoustic recognition module is used to collect the target sound source information, and the audio signal can be preprocessed, such as framing, windowing, pre-emphasis, etc. The collected sound source information is feature extracted. Common sound source features include MFCC (Mel frequency cepstral coefficient) features. The librosa library can be used to extract these features, and the extracted features are input into the core control module. The core control module contains a deep learning model for further analyzing and extracting the target sound source features.

[0060] The core control module compares the extracted target sound source features with historical sound source features, which can be stored in a database for matching and identification. Using machine learning algorithms, such as deep learning models, the target sound source features are matched with historical sound source features to determine the sound source type. Once the target sound source type is determined, the core control module looks for historical sample pose information that matches the sound source type. Based on the sound source type and historical data, the target pose is determined.

[0061] In a more preferred embodiment, step S20 further includes:

[0062] Extracting the target sound source frequency of the target sound source information through the core control module;

[0063] Determining a target sound source type of the target sound source information based on the target sound source frequency and the historical sound source frequency;

[0064] The sample pose that matches the target sound source type and the target sound source frequency in the historical sound source sample information is determined as the target pose.

[0065] In a specific embodiment, an acoustic recognition module is started to collect audio signals of a target sound source, and the collected audio signals are preprocessed, including denoising, normalization and other steps. Fourier transform (FFT) or other spectrum analysis methods are used to convert the audio signals from the time domain to the frequency domain to extract the frequency information of the target sound source, and the main frequency components of the target sound source are identified and extracted. These components may be fundamental frequencies or harmonic frequencies, depending on the characteristics of the sound source.

[0066] The extracted target sound source frequency is compared with the historical sound source frequency database. The historical sound source frequency database should contain the frequency characteristics of known sound source types. The target sound source frequency is classified using pattern recognition or machine learning algorithms, such as support vector machines (SVM) or neural networks, to determine the sound source type. Based on the characteristics of the target sound source frequency, it is associated with the sound source type that best matches in the historical data to determine the type of the target sound source.

[0067] Once the target sound source type is determined, the historical sound source sample information database is queried to find the sample pose information that matches the target sound source type and frequency. If there are multiple matching sample poses, they can be further screened based on the similarity of the target sound source frequency or other relevant features (such as the intensity and duration of the sound source). The best matching sample pose is determined as the target pose. The pose information can be the coordinates of the sound source in space, or the angle and distance relative to a reference point.

[0068] The determined target posture information is output to the motor drive module to control the visual recognition module to move to the corresponding position. At the same time, the target posture information can also be output to other systems or users for further analysis or decision-making.

[0069] based on Figure 1 In the embodiment shown, step S20 in this embodiment further includes:

[0070] The visual recognition module is driven to move to the target position by the servo motor;

[0071] When the visual recognition module reaches the target position, the camera is adjusted to the target posture through the servo motor.

[0072] In a specific embodiment, the core control module receives the coordinate information of the target position, which can be obtained from a sound source positioning system or other sensors. The core control module calculates the path from the current position of the visual recognition module to the target position. The core control module sends a movement instruction to the servo motor, and the instruction includes the distance, direction and speed of movement. After receiving the instruction, the servo motor starts to drive the visual recognition module (such as a camera) to move along the calculated path. During the movement, the servo motor may be equipped with an encoder or other position feedback device to monitor the position of the visual recognition module in real time.

[0073] After the visual recognition module reaches the target position, the servo motor stops moving, and the core control module receives the target posture information, which is obtained through a preset angle or calculated by the visual recognition algorithm. The core control module calculates the difference between the current posture of the camera and the target posture, and determines the adjustment parameters.

[0074] The core control module sends a posture adjustment instruction to the servo motor, and the servo motor adjusts the posture of the camera according to the instruction. After the camera is adjusted to the target posture, the servo motor stops the posture adjustment action.

[0075] based on Figure 1 In the embodiment shown, before step S30, the following steps are also included:

[0076] Under the condition of controlling the visual recognition module to the target posture, determining whether the target visual information is collected by the visual recognition module;

[0077] If the target visual information is not collected by the target visual recognition module, the target position and posture of the target sound source information is adjusted again by the core control module.

[0078] Based on any of the above embodiments, in this embodiment, before step S10, the following steps are included:

[0079] Starting the pan / tilt fusion control system with integrated acoustic positioning;

[0080] Determining whether the target sound source information is collected by the acoustic recognition module;

[0081] If the target sound source information is not collected by the acoustic recognition module, the position of the pan / tilt fusion control system with integrated acoustic positioning is adjusted to collect the target sound source information again.

[0082] See also Figure 6 , Figure 6 The embodiment of the present application provides a schematic block diagram of a pan-tilt fusion control device with integrated acoustic positioning, and the pan-tilt fusion control device with integrated acoustic positioning is used to execute the aforementioned pan-tilt fusion control method with integrated acoustic positioning. The pan-tilt fusion control device with integrated acoustic positioning can be configured on a server.

[0083] like Figure 6 As shown, the pan-tilt fusion control device 400 with integrated acoustic positioning includes:

[0084] A target sound source information acquisition module 410, configured to acquire target sound source information through the acoustic recognition module, and transmit the target sound source information to the core control module through the communication module;

[0085] A target posture adjustment module 420 is used to parse the target sound source information through the core control module, determine the target posture, and control the visual recognition module to the target posture through the motor drive module;

[0086] The fusion control information generating module 430 is used to collect the target visual information through the visual recognition module, and fuse the target visual information with the target sound source information to generate fusion control information.

[0087] Furthermore, the pan / tilt fusion control device 400 with integrated acoustic positioning includes:

[0088] A historical sound source sample information acquisition module is used to acquire historical sound source sample information;

[0089] A historical sound source sample information extraction module, used for parsing the historical sound source sample information through the core control module, and extracting the historical sound source features and / or historical sound source frequencies of the historical sound source sample information;

[0090] The sample pose determination module is used to determine the sound source type of the historical sound source sample information according to each of the historical sound source features and / or the historical sound source frequency, and determine the sample pose of the historical sound source sample information according to the sound source type of the historical sound source sample information.

[0091] Furthermore, the target posture adjustment module 420 includes:

[0092] A target sound source feature extraction unit, used to extract the target sound source features of the target sound source information through the core control module;

[0093] a target sound source type determination unit, configured to determine a target sound source type of the target sound source information according to the target sound source feature and the historical sound source feature;

[0094] The target posture determination unit is used to determine the sample posture corresponding to the sound source type matching the target sound source type in the historical sound source sample information as the target posture.

[0095] Furthermore, the target posture adjustment module 420 includes:

[0096] A target sound source frequency extraction unit, used to extract the target sound source frequency of the target sound source information through the core control module;

[0097] a target sound source type determining unit, configured to determine a target sound source type of the target sound source information based on the target sound source frequency and the historical sound source frequency;

[0098] The target posture determination unit is used to determine the sample posture that matches the target sound source type and the target sound source frequency in the historical sound source sample information as the target posture.

[0099] Furthermore, the target posture adjustment module 420 includes:

[0100] A visual recognition module moving unit, used for driving the visual recognition module to move to the target position through the servo motor;

[0101] A target posture adjustment unit is used to adjust the camera to the target posture through the servo motor when the visual recognition module reaches the target position.

[0102] Furthermore, the pan / tilt fusion control device 400 with integrated acoustic positioning includes:

[0103] A target visual information collection and judgment module, used to judge whether the target visual information is collected by the visual recognition module when controlling the visual recognition module to the target posture;

[0104] The target posture adjustment module is used to adjust the target posture of the target sound source information again through the core control module if the target visual information is not collected by the target visual recognition module.

[0105] Furthermore, the pan / tilt fusion control device 400 with integrated acoustic positioning includes:

[0106] A pan-tilt fusion control system startup module, used to start the pan-tilt fusion control system with integrated acoustic positioning;

[0107] A target sound source information collection and judgment module, used to judge whether the target sound source information is collected by the acoustic recognition module;

[0108] The target sound source information re-collection module is used to adjust the position of the pan-tilt fusion control system with integrated acoustic positioning and re-collect the target sound source information if the target sound source information is not collected by the acoustic recognition module.

[0109] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0110] The above-mentioned device can be implemented in the form of a computer program. Figure 7 Runs on the computer device shown.

[0111] See also Figure 7 , Figure 7 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0112] See also Figure 7 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0113] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any one of the pan-tilt fusion control methods with integrated acoustic positioning.

[0114] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0115] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any pan-tilt fusion control method with integrated acoustic positioning.

[0116] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0117] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0118] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0119] Collecting target sound source information through the acoustic recognition module, and transmitting the target sound source information to the core control module through the communication module;

[0120] Parsing the target sound source information through the core control module to determine the target posture, and controlling the visual recognition module to the target posture through the motor drive module;

[0121] The target visual information is collected through the visual recognition module, and the target visual information is fused with the target sound source information to generate fusion control information.

[0122] In one embodiment, the target sound source information is parsed by the core control module to determine the target posture, and the visual recognition module is controlled by the motor drive module to be before the target posture, so as to achieve:

[0123] Obtain historical sound source sample information;

[0124] Parsing the historical sound source sample information by the core control module to extract historical sound source features and / or historical sound source frequencies of the historical sound source sample information;

[0125] The sound source type of the historical sound source sample information is determined according to each of the historical sound source features and / or the historical sound source frequency, and the sample position and pose of the historical sound source sample information is determined according to the sound source type of the historical sound source sample information.

[0126] In one embodiment, the target sound source information is parsed by the core control module to determine the target position and posture, so as to achieve:

[0127] Extracting target sound source features of the target sound source information through the core control module;

[0128] Determining a target sound source type of the target sound source information according to the target sound source feature and the historical sound source feature;

[0129] The sample pose corresponding to the sound source type matching the target sound source type in the historical sound source sample information is determined as the target pose.

[0130] In one embodiment, the core control module parses the target sound source information to determine the target position and posture, and is also used to achieve:

[0131] Extracting the target sound source frequency of the target sound source information through the core control module;

[0132] Determining a target sound source type of the target sound source information based on the target sound source frequency and the historical sound source frequency;

[0133] The sample pose that matches the target sound source type and the target sound source frequency in the historical sound source sample information is determined as the target pose.

[0134] In one embodiment, the motor driving module controls the visual recognition module to the target posture to achieve:

[0135] The visual recognition module is driven to move to the target position by the servo motor;

[0136] When the visual recognition module reaches the target position, the camera is adjusted to the target posture through the servo motor.

[0137] In one embodiment, the visual recognition module collects target visual information, and fuses the target visual information with the target sound source information before generating fusion control information, which is used to achieve:

[0138] Under the condition of controlling the visual recognition module to the target posture, determining whether the target visual information is collected by the visual recognition module;

[0139] If the target visual information is not collected by the target visual recognition module, the target position and posture of the target sound source information is adjusted again by the core control module.

[0140] In one embodiment, the target sound source information is collected by the acoustic recognition module, and before the target sound source information is transmitted to the core control module by the communication module, it is used to achieve:

[0141] Starting the pan / tilt fusion control system with integrated acoustic positioning;

[0142] Determining whether the target sound source information is collected by the acoustic recognition module;

[0143] If the target sound source information is not collected by the acoustic recognition module, the position of the pan / tilt fusion control system with integrated acoustic positioning is adjusted to collect the target sound source information again.

[0144] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the pan-tilt fusion control methods with integrated acoustic positioning provided in the embodiments of the present application.

[0145] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Medi a Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card), etc., equipped on the computer device.

[0146] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A pan-tilt fusion control method integrating acoustic positioning, characterized in that: The pan-tilt fusion control method with integrated acoustic positioning is applied to a pan-tilt fusion control system with integrated acoustic positioning. The pan-tilt fusion control system with integrated acoustic positioning includes an acoustic recognition module, a communication module, a core control module, a motor drive module and a visual recognition module. The pan-tilt fusion control method with integrated acoustic positioning includes: Collecting target sound source information through the acoustic recognition module, and transmitting the target sound source information to the core control module through the communication module; Parsing the target sound source information through the core control module to determine the target posture, and controlling the visual recognition module to the target posture through the motor drive module; The target visual information is collected through the visual recognition module, and the target visual information is fused with the target sound source information to generate fusion control information.

2. The pan-tilt fusion control method with integrated acoustic positioning according to claim 1, characterized in that: The method of parsing the target sound source information through the core control module, determining the target posture, and controlling the visual recognition module to the target posture through the motor drive module includes: Obtain historical sound source sample information; Parsing the historical sound source sample information by the core control module to extract historical sound source features and / or historical sound source frequencies of the historical sound source sample information; The sound source type of the historical sound source sample information is determined according to each of the historical sound source features and / or the historical sound source frequency, and the sample position and pose of the historical sound source sample information is determined according to the sound source type of the historical sound source sample information.

3. The pan-tilt fusion control method with integrated acoustic positioning according to claim 2, characterized in that: The step of parsing the target sound source information through the core control module to determine the target position and posture includes: Extracting target sound source features of the target sound source information through the core control module; Determining a target sound source type of the target sound source information according to the target sound source feature and the historical sound source feature; The sample pose corresponding to the sound source type matching the target sound source type in the historical sound source sample information is determined as the target pose.

4. The pan-tilt fusion control method with integrated acoustic positioning according to claim 2, characterized in that: The step of analyzing the target sound source information by the core control module to determine the target position and posture also includes: Extracting the target sound source frequency of the target sound source information through the core control module; Determining a target sound source type of the target sound source information based on the target sound source frequency and the historical sound source frequency; The sample pose that matches the target sound source type and the target sound source frequency in the historical sound source sample information is determined as the target pose.

5. The pan-tilt fusion control method with integrated acoustic positioning according to claim 1, characterized in that: The motor drive module includes at least one servo motor, the visual recognition module includes at least one camera, the target posture includes a target position and a target posture, and controlling the visual recognition module to the target posture through the motor drive module includes: The visual recognition module is driven to move to the target position by the servo motor; When the visual recognition module reaches the target position, the camera is adjusted to the target posture through the servo motor.

6. The pan-tilt fusion control method with integrated acoustic positioning according to claim 1, characterized in that: The method includes collecting target visual information through the visual recognition module, fusing the target visual information with the target sound source information, and generating fusion control information, including: Under the condition of controlling the visual recognition module to the target posture, determining whether the target visual information is collected by the visual recognition module; If the target visual information is not collected by the target visual recognition module, the target position and posture of the target sound source information is adjusted again by the core control module.

7. The pan-tilt fusion control method of integrated acoustic positioning according to any one of claims 1 to 6, characterized in that: The step of collecting target sound source information through the acoustic recognition module and transmitting the target sound source information to the core control module through the communication module includes: Starting the pan / tilt fusion control system with integrated acoustic positioning; Determining whether the target sound source information is collected by the acoustic recognition module; If the target sound source information is not collected by the acoustic recognition module, the position of the pan / tilt fusion control system with integrated acoustic positioning is adjusted to collect the target sound source information again.

8. A pan-tilt fusion control device with integrated acoustic positioning, characterized in that: include: a target sound source information acquisition module, used to acquire target sound source information through the acoustic recognition module, and transmit the target sound source information to the core control module through the communication module; A target posture adjustment module, used for parsing the target sound source information through the core control module, determining the target posture, and controlling the visual recognition module to the target posture through the motor drive module; The fusion control information generating module is used to collect the target visual information through the visual recognition module, and fuse the target visual information with the target sound source information to generate fusion control information.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the pan-tilt fusion control method for integrated acoustic positioning as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the pan-tilt fusion control method for integrated acoustic positioning according to any one of claims 1 to 7.

Images