Target object positioning method, device, electronic device and storage medium

Abstract

The embodiments of the present application provide a method, device, electronic device and storage medium for positioning a target object, which relates to the field of computer technology and can improve the accuracy of positioning a target object in the process of using video and audio to coordinate positioning. The method includes: determining the first position information of each first object in at least one first object in a first coordinate system according to the acquired first information; determining the second position information of each first object in a second coordinate system according to a mapping relationship and the first position information of each first object, wherein the first coordinate system is different from the second coordinate system; determining the third position information of each second object in at least one second object in the second coordinate system according to the acquired second information; determining the target object according to the third position information of each second object and the second position information of each first object, so as to locate the target object, wherein the target object is the object corresponding to the third position information when the second position information matches the second position information.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device and storage medium for locating a target object. Background Art

[0002] With the development of computer technology, target positioning has become increasingly important as an important part of the visual field. At present, target positioning technology based on computer vision is gradually changing people's lifestyles, especially in human-computer interaction, and is widely used in monitoring, medical imaging, unmanned driving, remote control, interactive games and other fields. In multimedia video conferencing, in order to effectively improve the user's viewing experience and bring good visual effects to users, the target object (for example: the speaker in the meeting) is located and the close-up of the target object is taken, which can bring users a better visual experience and conference experience.

[0003] At present, the target object can be located by combining video (also called image) and audio. Compared with the method of using only audio positioning (also called sound source positioning), this solution can effectively filter out the influence of other noises and improve the positioning accuracy. However, in the process of using video and audio for positioning, the reference coordinate system used for video positioning may be different from the reference coordinate system used for audio positioning, so that the position information of the target object determined by video positioning is different from the position information of the target object determined by audio positioning. The two cannot be used together, and the target object cannot be accurately located. Summary of the invention

[0004] The present application provides a method, device, electronic device and storage medium for positioning a target object. In the process of using video and audio for positioning, when the reference coordinate systems used by video positioning and audio positioning are different, the position information of the target object determined by video positioning and the position information of the target object determined by audio positioning can be used in combination to improve the accuracy of positioning the target object.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a method for positioning a target object, the method comprising: determining first position information of each first object in at least one first object in a first coordinate system based on acquired first information; determining second position information of each first object in a second coordinate system based on a mapping relationship and the first position information of each first object, the mapping relationship being the relationship between the first position information of the object and the second position information of the object, the first coordinate system being different from the second coordinate system; determining third position information of each second object in at least one second object in the second coordinate system based on the acquired second information; determining a target object based on the third position information of each second object and the second position information of each first object, so as to locate the target object, the target object being an object corresponding when the third position information matches the second position information; wherein the first information is image information, and the second information is audio information; or, the first information is audio information, and the second information is image information.

[0007] Based on the above technical solution, after the first position information of each first object in the first coordinate system is determined using the first information, it is converted into the second position information of each first target object in the second coordinate system according to the mapping relationship, and the first coordinate system is different from the second coordinate system. Then, the third position information of each second object in the second coordinate system is determined using the second information, and the target object to be tracked can be determined according to the second position information of each first object and the third position information of each second object, so as to achieve the tracking of the target object. In the process of using video and audio for positioning, when the reference coordinate systems used for video positioning and audio positioning are different, the position information of the target object determined by video positioning and the position information of the target object determined by audio positioning can be used in combination, so as to improve the accuracy of positioning the target object.

[0008] In a possible implementation, the first position information is the first horizontal azimuth of the object in the first coordinate system, and the second position information is the second horizontal azimuth of the object in the second coordinate system; determining the second position information of each first object in the second coordinate system according to the mapping relationship and the first position information of each first object includes: determining the second horizontal azimuth of each first object according to the first horizontal azimuth of each first object, z and / or x corresponding to each first object, and y corresponding to each first object; wherein, if the first information is image information and the second information is audio information, x is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system, y is the vertical distance between the first object and the origin of the first coordinate system or the origin of the second coordinate system, and z is the horizontal distance between the first object and the origin of the second coordinate system; if the first information is audio information and the second information is image information, x is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system, y is the vertical distance between the first object and the origin of the first coordinate system or the origin of the second coordinate system, and z is the horizontal distance between the first object and the origin of the first coordinate system.

[0009] In a possible implementation, the value of y is determined according to the imaging size of the target area of ​​the object in the acquired image information.

[0010] In a possible implementation, the value of y is negatively correlated with the imaging size of the target area of ​​the object in the acquired image information.

[0011] In a possible implementation, the value of y and the imaging size of the target area of ​​the object satisfy the following formula: y = am d , where a is a preset constant greater than 0, d is a preset constant less than 0, and m is the imaging size of the target area of ​​the object.

[0012] In one possible implementation, z satisfies the following formula:

[0013] In a possible implementation, the mapping relationship satisfies the following formula:

[0014]

[0015] Wherein, if the first information is image information and the second information is audio information, α1 is the first horizontal azimuth, and α1 is greater than 90°, and β1 is the second horizontal azimuth. If the first information is audio information and the second information is image information, β1 is the first horizontal azimuth, α1 is the second horizontal azimuth, and α1 is greater than 90°.

[0016] In a possible implementation, the mapping relationship satisfies the following formula:

[0017]

[0018] Among them, if the first information is image information and the second information is audio information, then α1 is the first horizontal azimuth angle, and α1 is less than 90°, and β1 is the second horizontal azimuth angle; if the first information is audio information and the second information is image information, then β1 is the first horizontal azimuth angle, α1 is the second horizontal azimuth angle, and α1 is less than 90°.

[0019] In a possible implementation, the mapping relationship satisfies the following formula:

[0020]

[0021] Among them, if the first information is image information and the second information is audio information, then α1 is the first horizontal azimuth angle, and α1 is equal to 90°, and β1 is the second horizontal azimuth angle; if the first information is audio information and the second information is image information, then β1 is the first horizontal azimuth angle, α1 is the second horizontal azimuth angle, and α1 is equal to 90°.

[0022] In a possible implementation, when the position of the object remains unchanged, the imaging size of the target area of ​​the object in the acquired image information is negatively correlated with the target height, and the target height is the distance between the origin of the first coordinate system or the origin of the second coordinate system and the reference surface. Based on this implementation, the imaging size of the object can be determined according to the actual value of the target height, that is, the height at which the target tracking device is actually installed, which can improve the accuracy of the determined y value and z value, thereby making the mapping relationship more accurate and improving the accuracy of positioning the target object.

[0023] In one possible implementation, the target height and y satisfy the following formula: Wherein, y1 is the distance between the object and the origin of the first coordinate system or the origin of the second coordinate system after being mapped to the target vertical plane, and h is the target height.

[0024] In a possible implementation, the value of y1 is determined according to the imaging size of the target area of ​​the object in the acquired image information.

[0025] In a possible implementation, the value of y1 is negatively correlated with the imaging size of the target area of ​​the object in the acquired image information.

[0026] In a possible implementation, the value of y1 and the imaging size of the target area of ​​the object satisfy the following formula: y1 = am d , where a is a preset constant greater than 0, d is a preset constant less than 0, and m is the imaging size of the target area of ​​the object.

[0027] In a second aspect, the present application provides a target tracking device, including an acquisition unit and a processing unit; the acquisition unit is used to acquire first information; the processing unit is used to determine the first position information of each first object in at least one first object in a first coordinate system based on the first information; the processing unit is also used to determine the second position information of each first object in a second coordinate system based on a mapping relationship and the first position information of each first object, the mapping relationship being the relationship between the first position information of the object and the second position information of the object, and the first coordinate system is different from the second coordinate system; the acquisition unit is also used to acquire second information; the processing unit is also used to determine the third position information of each second object in the second coordinate system based on the second information; the processing unit is also used to determine the target object based on the third position information of each second object and the second position information of each first object to locate the target object, the target object being the object corresponding when the third position information matches the second position information; wherein the first information is image information, and the second information is audio information; or, the first information is audio information, and the second information is image information.

[0028] In a possible implementation, the first position information is the first horizontal azimuth of the object in the first coordinate system, and the second position information is the second horizontal azimuth of the object in the second coordinate system; the processing unit is specifically used to determine the second horizontal azimuth of each first object according to the first horizontal azimuth of each first object, z and / or x corresponding to each first object, and y corresponding to each first object; wherein, if the first information is image information and the second information is audio information, x is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system, y is the vertical distance between the first object and the origin of the first coordinate system or the origin of the second coordinate system, and z is the horizontal distance between the first object and the origin of the second coordinate system; if the first information is audio information and the second information is image information, x is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system, y is the vertical distance between the first object and the origin of the first coordinate system or the origin of the second coordinate system, and z is the horizontal distance between the first object and the origin of the first coordinate system.

[0029] In a possible implementation, the value of y is determined according to the imaging size of the target area of ​​the object in the acquired image information.

[0030] In a possible implementation, the value of y is negatively correlated with the imaging size of the target area of ​​the object in the acquired image information.

[0031] In a possible implementation, the value of y and the imaging size of the target area of ​​the object satisfy the following formula: y = am d , where a is a preset constant greater than 0, d is a preset constant less than 0, and m is the imaging size of the target area of ​​the object.

[0032] In one possible implementation, z satisfies the following formula:

[0033] In a possible implementation, the mapping relationship satisfies the following formula:

[0034]

[0035] Wherein, if the first information is image information and the second information is audio information, α1 is the first horizontal azimuth, and α1 is greater than 90°, and β1 is the second horizontal azimuth. If the first information is audio information and the second information is image information, β1 is the first horizontal azimuth, α1 is the second horizontal azimuth, and α1 is greater than 90°.

[0036] In a possible implementation, the mapping relationship satisfies the following formula:

[0037]

[0038] Among them, if the first information is image information and the second information is audio information, then α1 is the first horizontal azimuth angle, and α1 is less than 90°, and β1 is the second horizontal azimuth angle; if the first information is audio information and the second information is image information, then β1 is the first horizontal azimuth angle, α1 is the second horizontal azimuth angle, and α1 is less than 90°.

[0039] In a possible implementation, the mapping relationship satisfies the following formula:

[0040]

[0041] Among them, if the first information is image information and the second information is audio information, then α1 is the first horizontal azimuth angle, and α1 is equal to 90°, and β1 is the second horizontal azimuth angle; if the first information is audio information and the second information is image information, then β1 is the first horizontal azimuth angle, α1 is the second horizontal azimuth angle, and α1 is equal to 90°.

[0042] In a possible implementation, when the position of the object remains unchanged, the imaging size of the target area of ​​the object in the acquired image information is negatively correlated with the target height, and the target height is the distance between the origin of the first coordinate system or the origin of the second coordinate system and the reference surface. Based on this implementation, the imaging size of the target area of ​​the object can be determined according to the actual value of the target height, that is, the height at which the target tracking device is actually installed, which can improve the accuracy of the determined y value and z value, thereby making the mapping relationship more accurate and improving the accuracy of positioning the target object.

[0043] In one possible implementation, the target height and y satisfy the following formula: Wherein, y1 is the distance between the object and the origin of the first coordinate system or the origin of the second coordinate system after being mapped to the target vertical plane, and h is the target height.

[0044] In a possible implementation, the value of y1 is determined according to the imaging size of the target area of ​​the object in the acquired image information.

[0045] In a possible implementation, the value of y1 is negatively correlated with the imaging size of the target area of ​​the object in the acquired image information.

[0046] In a possible implementation, the value of y1 and the imaging size of the target area of ​​the object satisfy the following formula: y1 = am d , where a is a preset constant greater than 0, d is a preset constant less than 0, and m is the imaging size of the target area of ​​the object.

[0047] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory is coupled to the processor; the memory is used to store computer program code, and the computer program code comprises computer instructions; when the computer instructions are executed by the processor, the processor executes the method described in the first aspect above and any possible implementation thereof.

[0048] In a fourth aspect, the present application provides a computer-readable storage medium, comprising computer instructions, which, when executed on a computing device, enable the computing device to execute the method as described in the first aspect and any possible implementation thereof.

[0049] In a fifth aspect, the present application provides a computer program product, which includes: a computer program or instructions, which, when the computer program or instructions are run on a computer, enable the computer to execute the method described in the first aspect above and any possible implementation thereof.

[0050] In a sixth aspect, the present application provides a chip system comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to at least one processor, and when the at least one processor executes the instructions, the at least one processor executes the method described in the first aspect above and any possible implementation method thereof.

[0051] It should be noted that the technical effects brought about by any possible implementation method in the above-mentioned second to sixth aspects can refer to the technical effects brought about by the corresponding implementation method in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram of the structure of a target object positioning device provided in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of the positions of the center point of a fixed-focus camera and the center point of a microphone array provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of a flow chart of a method for locating a target object provided in an embodiment of the present application;

[0055] Figure 4a A schematic diagram of the positions of an object and a target object positioning device provided in an embodiment of the present application;

[0056] Figure 4b A schematic diagram of the relationship between position information of an object in different coordinate systems provided in an embodiment of the present application;

[0057] Figure 4c A schematic diagram of the relationship between position information of another object in different coordinate systems provided in an embodiment of the present application;

[0058] Figure 4d A schematic diagram of the relationship between position information of another object in different coordinate systems provided in an embodiment of the present application;

[0059] Figure 5 A schematic diagram of the relationship between the distance from an object to a target object positioning device and the imaging size of the target area of ​​the object provided in an embodiment of the present application;

[0060] Figure 6 A schematic diagram of the relationship between the height of a target object positioning device and the imaging size of the target area of ​​the object provided in an embodiment of the present application;

[0061] Figure 7 A schematic diagram of errors existing at different target heights provided in an embodiment of the present application;

[0062] Figure 8 A schematic diagram of the structure of a target object positioning device provided in an embodiment of the present application;

[0063] Fig. 9 A schematic diagram of the structure of another target object positioning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0065] The terms "including" and "having" and any variations thereof mentioned in the description of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0066] In the description of this application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "plurality" means two or more. The words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not limit them to be different.

[0067] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0068] First, to facilitate understanding, the technical terms and concepts that may be involved in this application are introduced below.

[0069] 1. Fixed-focus lens

[0070] A fixed-focus lens is a lens with only one fixed focal length, only one focal length, or only one field of view, which can be installed on a camera. A fixed-focus lens does not have a zoom function. The biggest advantage of a fixed-focus lens over a zoom lens is that it has a fast focusing speed and stable image quality.

[0071] 2. Panoramic lens

[0072] Panoramic lenses use panoramic technology to obtain a 360° horizontal field of view and a certain angle vertical field of view. Its imaging method can provide all-round information about objects and environments in real time, buying time for subsequent image processing and analysis. Panoramic lenses generally use fisheye lenses, which have a very wide viewing angle of up to 180 degrees.

[0073] 3. Microphone array

[0074] A microphone array is a system composed of a certain number of acoustic sensors (usually microphones) used to sample and process the control characteristics of the sound field. A microphone array can also be called an array microphone. After the microphone array is arranged according to the specified requirements, it can solve many room acoustic problems with the corresponding algorithms, such as sound source localization, reverberation, speech enhancement, blind source separation, etc. Among them, sound source localization refers to the use of a microphone array to calculate the angle and distance of the target speaker, thereby achieving the positioning of the target speaker and subsequent directional voice pickup. It is a very important pre-processing technology in the fields of human-computer interaction, audio and video conferencing, etc. Using a microphone array, 180° / 360° voice signal acquisition can be achieved, and the direction of the target speaker can be determined by sound source localization. Microphone arrays include linear arrays, circular arrays, spherical arrays, etc.

[0075] Figure 1 The following is a schematic diagram of a device for locating a target object provided in an embodiment of the present application. Figure 1 As shown, the target object positioning device 100 includes an image collector 101, an audio collector 102 and a processor. Figure 1 Not shown in FIG.

[0076] Among them, the image collector 101 can be used to collect image information, and the image information can refer to image information in the environment where the target object positioning device 100 is located, including but not limited to people, objects, etc. For example: the image collector can be a fixed-focus camera, and a fixed-focus camera is a camera with a fixed-focus lens installed. At the same time, the fixed-focus camera can also be a panoramic camera, that is, the fixed-focus lens can also be a panoramic lens. The fixed-focus camera can be used for video image shooting to locate the object using image information. Of course, the image collector 101 can also be other types of devices that can be used to collect image information, and this application does not make specific limitations on this.

[0077] The audio collector 102 is used to collect audio information, and the audio information may refer to the sound information in the environment where the target object positioning device is located, including but not limited to the sound of people and objects. For example, the audio collector 102 may be a microphone array. The microphone array may be used to collect audio information for sound source positioning. The audio collector 102 may also be other types of devices that can be used to collect audio information, and this application does not specifically limit this.

[0078] It should be pointed out that Figure 1 The target object positioning device shown is based on the example that the center point of the image collector 101 is the center of the circle formed by the lens, namely O1, the center point of the audio collector 102 is the midpoint of the line connecting the elements in the linear microphone array, namely O2, and O1 and O2 are in the same horizontal plane. Figure 1The positions of the image collector 101 and the audio collector 102 shown are merely illustrative. In actual applications, the image collector 101 and the audio collector 102 may have other positions. For example, the image collector 101 may be placed on the right side of the audio collector 102. The position of the image collector 101 and / or the position of the audio collector 102 may be moved up and down. The center point of the image collector 101 and the center point of the audio collector 102 may not be on the same horizontal plane.

[0079] The processor includes one or more central processing units (CPUs). The CPU may be a single-core CPU (single-CPU) or a multi-core CPU (multi-CPU). The processor may locate the object using the image information collected by the image collector 101, may locate the object using the audio information collected by the audio collector 102, or may locate the object using the image information collected by the image collector 101 and the audio information collected by the audio collector 102.

[0080] Optionally, the target object positioning device 100 further includes a memory, which includes but is not limited to a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), a flash memory, or an optical memory, etc. The memory is used to store a computer program code, which can be executed by a processor to realize the positioning of the object.

[0081] Understandably, Figure 1 The illustrated structure does not constitute a specific limitation on the target object positioning device 100. In other embodiments of the present application, the target object positioning device 100 may include more or fewer components than those shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0082] Combine the following Figure 1The target object positioning device shown illustrates the process of positioning by video and audio. The following text uniformly introduces the video collector as a fixed-focus camera and the audio collector as a microphone array. Taking the multimedia video conferencing scene as an example, the image information is collected by a fixed-focus camera, and the image information includes but is not limited to people, objects, etc., and then the object is determined based on the collected image information. Taking the object as an example, the target object positioning device can detect the object from the image information through a deep learning algorithm (also known as personnel target detection), for example: by identifying the head, face, shoulders, etc. of the object, the object is detected, and then the position information of the object (for example: azimuth, etc.) is calculated based on the position of the object in the image. Audio information is collected by a microphone array, and the audio information includes but is not limited to human voices, sounds emitted by other devices, etc. The sound source is located based on the acquired audio information, and the target object (for example: conference speaker) can be determined in combination with the object's position information. Optionally, after the target object is determined, the image output control of the target object can also be performed, for example: close-up of the target object, etc.

[0083] Compared with the method of positioning based on audio alone, the method of using video and audio to coordinate positioning can effectively filter out the influence of other noises. It should be pointed out that the video can be used to detect the object first, and then the audio can be used to locate the sound source; or the audio can be used to locate the sound source first, and then the video can be used to detect the object. It can be understood that the method of first using video to detect the object and then using audio to locate the sound source can effectively reduce the calculation area of ​​the sound source positioning and improve the positioning accuracy.

[0084] Among them, Figure 1 As shown, the origin of the coordinate system used when locating an object using video is usually the center point of the fixed-focus camera, that is, O1, and the origin of the coordinate system used when locating an object using audio is usually the center point of the microphone array, that is, O2. Due to the structural layout of the target object positioning device, or the inclusion of other devices that affect the structural layout of the target object positioning device (for example, audio, etc.), the center of the fixed-focus camera and the center of the microphone array do not coincide, that is, the origin of the coordinate system used is different, and thus the coordinate system used is also different, so that the position information (for example, azimuth) of the target object located by video is different from the position information of the target object located by audio, and the two cannot be used together, and the target object cannot be accurately located.

[0085] Figure 2 A schematic diagram showing the relative positions of the center point of the fixed-focus camera and the center point of the microphone array is shown. It can be understood that Figure 2The coordinate system shown is intended to illustrate that the positions of O1 and O2 are different. It can be a spatial rectangular coordinate system with the center point of the target object positioning device as a coordinate origin, or a coordinate system with other points as the coordinate origin, or other types of coordinate systems. Figure 2 As shown in the figure, O1 is the center point of the fixed-focus camera, O2 is the center point of the microphone array, where α1 is the angle of the object located by video, and β1 is the angle of the object located by audio, and α1 is different from β1. In the process of using video and audio for positioning, after the angle of each object (for example, each participant) is located by video, the angle of one of the objects is located by audio within the angle range located by video. For the same object, the angle located by video and the angle located by audio must be the same to achieve accurate positioning of the object. For example, taking the positioning of the speaker as an example, it is assumed that the azimuths of the three participants are first located by video, which are 15°, 35°, and 120°, respectively, where 35° is the azimuth of the speaker. Then the azimuths of the two sound sources are located by audio, which are 60° and 90°, respectively, where 90° is the azimuth located based on the human voice. Since the azimuth located by audio needs to be matched within the range of the azimuth located by video in order to determine the azimuth of the speaker, for the speaker, the azimuth located by video (i.e. 35°) does not match the azimuth located by audio (i.e. 90°), so the speaker cannot be accurately located.

[0086] In order to solve the above technical problems, the embodiment of the present application provides a method for positioning a target object. After the position information of each object is positioned by video (or audio), the position information of each object is converted into the position information in the coordinate system used for positioning by audio (or video) according to a pre-established mapping relationship, and then the position information positioned by audio (or video) is matched with the converted position information of each object. If the match is successful, the target object to be positioned can be determined to achieve the positioning of the target object. In the process of positioning by video and audio, when the reference coordinate systems used for video positioning and audio positioning are different, the position information of the target object determined by video positioning and the position information of the target object determined by audio positioning can be used in combination to improve the accuracy of positioning the target object.

[0087] like Figure 3 As shown, a method for locating a target object provided in an embodiment of the present application can be applied to Figure 1 The target object positioning device shown can also be applied to other devices, such as a server, etc. The method includes the following steps:

[0088] S301. Determine first position information of each first object in at least one first object in a first coordinate system according to acquired first information.

[0089] Among them, the first information can be image information or audio information, and the acquired image information can be taken by the target object positioning device itself, for example, image information taken by a fixed-focus camera. It can also be obtained from other devices, for example, image information is taken by other devices and then sent to the target object positioning device (or server). Optionally, the image information can be contained in a video or in a photo. Exemplarily, in the case where the image information is contained in a photo (i.e., a single image), the first position information of each object can be directly determined based on the image information. In the case where the image information is contained in a video (i.e., multiple images), for an object, the first position information of the object can be determined based on each frame of the image contained in the video, and then the final value of the first position information can be determined based on the multiple first position information determined. The final value can be a statistical value such as the mode, median, mean, etc., or it can be other values, thereby improving the positioning accuracy.

[0090] Exemplarily, the image information may refer to the image information of some or all objects within the range that can be captured by the camera, and the objects include but are not limited to people, objects, etc. In combination with the multimedia video conferencing scenario, taking the object as the participant as an example, the acquired image information is analyzed by an algorithm, and the algorithm may be a deep learning algorithm or other machine learning algorithm to determine the first position information of each participant.

[0091] It should be pointed out that the present application does not limit the type of camera used to capture image information. It can be a fixed-focus lens, zoom lens, panoramic lens, standard lens, super telephoto lens, medium focal length lens, macro lens, fisheye lens, etc.

[0092] The first position information is the position information of the object in the first coordinate system. The first coordinate system is the coordinate system used when determining the position information of the object according to the acquired first information, that is, the coordinate system used when locating the object using the first information. For example: Figure 1 The spatial rectangular coordinate system with O1 as the origin of the coordinate system, with the positive direction of the y-axis pointing vertically upward, the positive direction of the x-axis pointing horizontally to the right, and the positive direction of the z-axis pointing horizontally toward the user. Optionally, the first position information may be angle information, for example: the horizontal azimuth of the object in the first coordinate system. In combination with the coordinate system with O1 as the origin, the horizontal azimuth refers to the angle between the line connecting the object and O1 and the x-axis after the object is mapped to the plane formed by the x-axis and the z-axis (i.e., the horizontal plane), such as Figure 2α1 shown, or the supplementary angle of α1. The first position information may also be the pitch angle of the object in the first coordinate system, etc. The first position information may also be coordinate information, and of course may also be other types of information used to indicate the position of the object, which are uniformly described here.

[0093] Optionally, the first coordinate system may be a rectangular coordinate system, a spherical coordinate system, a cylindrical coordinate system, a polar coordinate system, etc. The present application does not limit the type of the first coordinate system.

[0094] S302: Determine second position information of each first object in a second coordinate system according to the mapping relationship and the first position information of each first object.

[0095] The second coordinate system is a coordinate system used when determining the position information of the object according to the second information, for example: Figure 1 The O2 point is the coordinate origin of the spatial rectangular coordinate system, with the vertical upward direction being the positive direction of the y-axis, the horizontal rightward direction being the positive direction of the x-axis, and the horizontal direction toward the user being the positive direction of the z-axis. Taking the second position information as angle information as an example, the angle information may be the horizontal azimuth of the object in the second coordinate system. Combined with the coordinate system with O2 as the origin, the horizontal azimuth refers to the angle between the line connecting the object and O2 and the x-axis after the object is mapped to the plane formed by the x-axis and the z-axis (i.e., the horizontal plane), as shown in FIG. Figure 2 β1 shown, or the supplementary angle of β1. Similar to the first position information, the second position information may also be other types of information used to indicate the position of the object.

[0096] The first coordinate system is different from the second coordinate system. Exemplarily, the first coordinate system and the second coordinate system are different including but not limited to different origins of the coordinate systems, the origins of the coordinate systems are not on the same vertical line, the types of the coordinate systems are different, the coordinate axes of the coordinate systems are arranged differently, etc.

[0097] Optionally, the second coordinate system may be a rectangular coordinate system, a spherical coordinate system, a cylindrical coordinate system, a polar coordinate system, etc., and the present application does not limit the type of the second coordinate system.

[0098] Wherein, when the first information is image information, the second information may be audio information; when the first information is audio information, the second information may be image information. Optionally, the acquired audio information may be collected by the target object positioning device itself, for example, audio information collected by a microphone array, or audio information collected by other sensors. It may also be acquired from other devices, for example, audio information collected by other devices and then sent to the target object positioning device (or server).

[0099] The mapping relationship is the relationship between the first position information of the object and the second position information of the object.

[0100] Optionally, the first position information is a first horizontal azimuth of the object in the first coordinate system, and the second position information is a second horizontal azimuth of the object in the second coordinate system; the second horizontal azimuth of each first object can be determined according to the first horizontal azimuth of each first object, z and / or x corresponding to each first object, and y corresponding to each first object;

[0101] Wherein, if the first information is image information and the second information is audio information, x is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system, y is the vertical distance between the first object and the origin of the first coordinate system or the origin of the second coordinate system, and z is the horizontal distance between the first object and the origin of the second coordinate system;

[0102] If the first information is audio information and the second information is image information, x is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system, y is the vertical distance between the first object and the origin of the first coordinate system or the origin of the second coordinate system, and z is the horizontal distance between the first object and the origin of the first coordinate system.

[0103] Optionally, the first position information is a first horizontal azimuth of the object in a first coordinate system, and the second position information is a second horizontal azimuth of the object in a second coordinate system. The mapping relationship may satisfy equation 1.1.

[0104]

[0105] Wherein, if the first information is image information and the second information is audio information, α1 is the first horizontal azimuth, β1 is the second horizontal azimuth, and z is the horizontal distance between the object and the origin of the second coordinate system. x is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system. Taking the origin of the first coordinate system as the center point O1 of the fixed-focus lens and the origin of the second coordinate system as the center point O2 of the microphone array as an example, assuming that O1 and O2 are on the same horizontal plane, x is the distance between the center point of the fixed-focus lens and the center point of the microphone array, that is, the distance between O1 and O2. The value of x is related to the structure of the target object positioning device, for example: it is related to the relative position of the fixed-focus lens and the microphone array in the target object positioning device. Optionally, the value of x can be preset by the developer. y is the vertical distance between the object and the origin of the first coordinate system or the origin of the second coordinate system, that is, the vertical distance between the object and the target object positioning device, that is, the vertical distance between the object and the installation plane of the target object positioning device.

[0106] If the first information is audio information and the second information is image information, β1 is the first horizontal azimuth, α1 is the second horizontal azimuth, z is the horizontal distance between the object and the origin of the first coordinate system, x is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system, and y is the vertical distance between the object and the origin of the first coordinate system or the origin of the second coordinate system.

[0107] Optionally, since the obtained parameters (eg, x, y, z, etc.) may have errors, the mapping relationship may also be expressed in the form of Formula 1.1'.

[0108]

[0109] Wherein, c is a preset constant, which can be used to eliminate the error of each parameter. It can be understood that the constants used to eliminate the error of each parameter can be the same, for example, all c, or different.

[0110] The above parameters can be determined by mapping the origin of the coordinate system and the target object to the same horizontal plane. For ease of understanding, the parameters in the above mapping relationship are explained below in conjunction with the accompanying drawings.

[0111] Combination Figure 1 The target object positioning device shown, such as Figure 4a As shown, plane A is the installation plane of the target object positioning device, plane B is the horizontal plane where the object is located, plane A and plane B may or may not be perpendicular to each other, and horizontal plane B is parallel to horizontal plane C. Among them, the x-axis (the x-axis in the coordinate system with O1 as the origin and the x-axis in the coordinate system with O2 as the origin) can be mapped to the x' axis on the horizontal plane B, O1 can be mapped to O1' on the horizontal plane B, and O2 can be mapped to O2' on the horizontal plane. α1 is the angle between the straight line connecting the object and O1' and the x' axis, which is called the first horizontal azimuth, such as α1; the angle between the straight line connecting the object and O2' and the x' axis is called the second horizontal azimuth, such as β1. Optionally, α1 can be Figure 4a The angle shown can also be the complementary angle corresponding to the angle, and β1 can also be Figure 4a The angle shown may also be the supplementary angle corresponding to the angle.

[0112] Combined with the coordinate system with O2 as the origin, the distance from the object to the plane formed by the y-axis and the z-axis is called the horizontal distance between the object and the origin of the second coordinate system, that is, z. The distance from the object to plane A (i.e., the plane formed by the x-axis and the y-axis) is called the vertical distance between the object and the origin of the first coordinate system or the origin of the second coordinate system, that is, y.

[0113] because Figure 4aIn the example shown, the origin of the first coordinate system and the origin of the second coordinate system are on the same horizontal plane C, so the distance between the origin of the first coordinate system and the origin of the second coordinate system is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system, that is, x. If the origin of the first coordinate system (for example, O1) and the origin of the second coordinate system (for example, O3) are not on the same horizontal plane, they can be mapped to the same horizontal plane (for example, horizontal plane C), and the distance between them is called the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system.

[0114] It should be noted that the mapping relationship may also have other representation forms, for example, the representation form of the relationship between the coordinates of an object in a first coordinate system and the coordinates of the object in a second coordinate system, which is not specifically limited in this application.

[0115] For example, Figure 4b A schematic diagram of first position information and second position information provided by an embodiment of the present application is shown. Wherein, O1 is the center point of the fixed-focus camera, and O2 is the center point of the microphone array. When the first information is image information and the second information is audio information, O1 is the origin of the first coordinate system, and O2 is the origin of the second coordinate system. When the first information is audio information and the second information is image information, O1 is the origin of the second coordinate system, and O2 is the origin of the first coordinate system.

[0116] Combination Figure 4b , according to the spatial relationship, we can get formula 1.2:

[0117]

[0118] According to the trigonometric function relationship, we can get formula 1.3

[0119] β1=α1-α 1.3

[0120] Equation 1.1 can be obtained by simplifying equation 1.2 and equation 1.3. Optionally, the mapping relationship can also be expressed by other trigonometric functions, including but not limited to tangent function, cosine function, etc., or other mathematical expressions, which can be converted into the representation provided in the embodiment of the present application, and are uniformly described here.

[0121] It should be noted that formula 1.1 is applicable when α1 is greater than 90°. When α1 is equal to 90° or α1 is less than 90°, formula 1.2 may be different. Based on the spatial relationship, the relationship between β1 and α1 can be obtained by the same logic when α1 is equal to 90° or α1 is less than 90°.

[0122] When α1 is equal to 90°, combined Figure 4c , according to the spatial relationship, we can get formula 1.4:

[0123]

[0124] Combination Figure 4c Since the horizontal distance z between the object and the origin of the second coordinate system is equal to the horizontal distance x between the origin of the first coordinate system and the origin of the second coordinate system, part or all of x in formula 1.4 can also be replaced by z.

[0125] Combining equations 1.3 and 1.4, we can get the relationship between β1 and α1, as shown in equation 1.5:

[0126]

[0127] When α1 is equal to 90°, the mapping relationship can be expressed as shown in Formula 1.5.

[0128] When α1 is less than 90°, the combination Figure 4d , according to the spatial relationship, we can get formula 1.6:

[0129]

[0130] Combining equation 1.3 and equation 1.6, we can get the relationship between β1 and α1, as shown in equation 1.7:

[0131]

[0132] When α1 is less than 90°, the mapping relationship can be expressed as shown in Formula 1.7.

[0133] Similarly, in order to eliminate the error, similar to the case where α1 is greater than 90°, when α1 is less than 90° and α1 is equal to 90°, the mapping relationship can also be expressed in a form similar to Formula 1.1'.

[0134] It should be noted that since α1 can be determined based on the acquired image information, the position of the object can be distinguished by the value of α1, or by using tanα1 and The difference is used to distinguish the position of the object.

[0135] It should be noted that, in the case of using video positioning first, the value of α1 can be determined based on the acquired image information, and in the case of using audio positioning first, the value of β1 can be determined based on the acquired audio information. The value of x is related to the structure of the target object positioning device and can be directly obtained based on the structure of the target object positioning device, but the value of y and the value of z cannot be directly obtained based on the structure of the target object positioning device or the relationship between the spatial positions. Therefore, the embodiment of the present application also provides a solution for determining the value of y and the value of z.

[0136] Optionally, the value of y can be determined based on the imaging size of the target area of ​​the object in the captured image information. It is understandable that the imaging size of the target area of ​​the object in the image information refers to the size of the imaging of the target area of ​​the object in the image information. Exemplarily, taking the object as a person as an example, the target area of ​​the object may refer to the area from the head of the person to the shoulders of the person, and the imaging size of the target area of ​​the object may refer to the size of the image of the area from the head of the person to the shoulders of the person. The target area may also be other areas, for example: the area from the head of the person to the waist of the person. The target area can be set by the developer according to actual needs, and this application does not limit this. Of course, the value of y can also be determined in other ways.

[0137] Optionally, the value of y is negatively correlated with the imaging size of the target area of ​​the object in the acquired image information.

[0138] For example, Figure 5 A schematic diagram showing a value of y and the imaging size of the target area of ​​the object is shown. Figure 5 As shown in the figure, the y values ​​corresponding to object 1, object 2, and object 3 are y1, y2, and y3, respectively, and the corresponding imaging sizes are size 1, size 2, and size 3, respectively. Therefore, by obtaining the imaging size of the target area of ​​the object at y1, y2, y3, ... yn, and then fitting the y value and the corresponding imaging size, the relationship between the y value and the imaging size of the target area of ​​the object can be obtained.

[0139] In a possible example, the value of y and the imaging size of the target area of ​​the object may satisfy Formula 1.8.

[0140] y=am d 1.8

[0141] Wherein, a is a preset constant greater than 0, d is a preset constant less than 0, and m is the imaging size of the target area of ​​the object.

[0142] In another possible example, the value of y and the imaging size of the target area of ​​the object may satisfy Formula 1.8'.

[0143] y=a'ln(m)+d' 1.8'

[0144] Wherein, a' and d' are both preset constants less than 0, and m is the imaging size of the target area of ​​the object.

[0145] In yet another possible example, the value of y and the imaging size of the target area of ​​the object may satisfy equation 1.8".

[0146] y=a”e d”m 1.8''

[0147] Wherein, a" is a preset constant greater than 0, d" is a preset constant less than 0, and m is the imaging size of the target area of ​​the object.

[0148] It should be pointed out that the present application does not limit the specific expression form of the relationship between the value of y and the imaging size of the target area of ​​the object, which can be expressed in various possible forms such as power function, exponential function, logarithmic function, etc.

[0149] Then, the value of z can be obtained according to the value of y, combined with Figure 4b , according to the spatial relationship, we can get formula 1.9.

[0150]

[0151] Similarly, combined Figure 4c , Figure 4d The spatial relationship shown can also determine the value of z based on the value of y, which will not be elaborated here.

[0152] S303: Determine third position information of each second object in the at least one second object in the second coordinate system according to the acquired second information.

[0153] The second object may be the same as or different from the first object. For the introduction of the second information, please refer to the above description, which will not be repeated here. Similar to the first position information, the third position information may be angle information or other types of information used to indicate the position of the object, which is not limited in this application.

[0154] S304: Determine a target object according to the third position information of each second object and the second position information of each first object to locate the target object.

[0155] Among them, the target object is the object corresponding to when the third location information matches the second location information. Optionally, when the third location information does not match the second location information, it can be determined that the target object does not currently exist. Taking a multimedia video conferencing scenario as an example, the target object may be a speaker, and the corresponding object when the third location information matches the second location information is the speaker, and the speaker can be located. When the third location information does not match the second location information, it means that there is currently no speaker, and no positioning is required. The target object may be one or more objects, and this application does not limit this.

[0156] The third position information of each second object is matched with the second position information of each first object. If the match is successful, the target object can be determined. Exemplarily, at least one third position information is determined based on the acquired audio information, that is, the third position information of different sound sources (for example, human voice, sound emitted by other devices, etc.) is determined respectively, and then the at least one third position information is matched with the second position information of each first object. If the match is successful (for example, the result of the match meets the preset condition, and the preset condition can be that the position information is exactly the same, or that the error in the position information meets the preset threshold range. Taking the position information as an angle as an example, the condition for successful matching can be that the angle is exactly the same, or that the angle error meets the preset angle range), then the object corresponding to the successfully matched position information is determined to be the target object, and the target object can be located according to its position information. Taking the target object as a speaker as an example, assuming that according to the mapping relationship, the second position information of participant 1 is 45°, the second position information of participant 2 is 60°, and the second position information of participant 3 is 150°, and then the third position information of sound source 1 determined according to the acquired audio information is 60°, and the third position information of sound source 2 is 90°. The two are matched. Assuming that the matching result requirements are exactly the same, it can be determined that the third position information of sound source 1 is the same as the second position information of participant 2, then participant 2 is the speaker, and 60° is the speaker's position information. Participant 2 can be located based on this position information.

[0157] Optionally, in practical applications, the imaging size of the target area of ​​the object is related to the target height, which is the distance between the origin of the first coordinate system or the origin of the second coordinate system and the reference surface, and can also be understood as the distance between the target object positioning device and the reference surface. The reference surface can be the ground or other standard surfaces.

[0158] Figure 6A schematic diagram of the imaging size of the target area of ​​an object under different target heights is shown. When the target height is h1, the imaging size of the target area of ​​the object is size 4, and the corresponding y value is y4. When the target height is h1+h2, the imaging size of the target area of ​​the object is size 5, and the corresponding y value is y5. y4 is smaller than y5, and size 4 is larger than size 5. That is, as the target height increases, the imaging size of the target area of ​​the object will decrease accordingly when the position of the object remains unchanged. That is, when the position of the object remains unchanged, the imaging size of the target area of ​​the object is negatively correlated. Therefore, due to the different target heights, the imaging size of the target area of ​​the object is also different, and thus the y value determined according to formula 1.8 is different. However, only when the target height is 0, the determined y value is the vertical distance between the object and the origin of the first coordinate system or the origin of the second coordinate system, that is, the target object positioning device. When the target height is not 0, the determined y value is the actual distance between the object and the target object positioning device, rather than the vertical distance between the object and the target object positioning device.

[0159] Therefore, due to the different target heights, the y value determined according to formula 1.8 may not be the vertical distance between the object and the target object positioning device, which may cause a certain error when converting the first position information into the second position information according to the mapping relationship. Figure 7 The figure shows the error magnitude at different target heights, where the horizontal axis represents the vertical distance between the object and the target object positioning device, and the vertical axis represents the error magnitude, wherein target height 1 is 0, target height 2 is 400, and target height 3 is 1000. Figure 7 It can be seen that the magnitude of the error is different at different target heights, and at the same target height, the smaller the vertical distance between the object and the target object positioning device, that is, the closer the object and the target object positioning device are, the larger the error. Moreover, the farther the distance between the origin of the first coordinate system and the origin of the second coordinate system is, the larger the error is.

[0160] Optional, combined Figure 6 , h1 is the position where the target height is 0, y4 (that is, y in Formula 2.0) is the vertical distance between the object and the target object positioning device, and Formula 2.0 can be obtained based on the spatial relationship.

[0161]

[0162] Among them, y1 (that is, Figure 6 y5) as the object, and the origin of the first coordinate system or the origin of the second coordinate system is mapped to the target vertical surface (for example: combined with Figure 4aAs shown, the distance between the vertical plane parallel to the plane formed by the y-axis and the z-axis, that is, the perpendicular line from the object to the straight line connecting O1' and O2', and the plane parallel to the vertical plane formed by the straight line connecting O2 and O2'). The vertical plane refers to the plane perpendicular to the horizontal plane. Optionally, the value of y1 is determined according to the imaging size of the target area of ​​the object, for example: calculated according to formula 1.8, h (that is Figure 6 h2) in the figure is the target height. Similar to the value of y, the value of y1 is also negatively correlated with the imaging size of the target area of ​​the object. At different target heights, according to formula 2.0, the vertical distance between the object and the target object positioning device can be determined.

[0163] Correspondingly, according to formula 2.0, combined with Figure 4b As well as Equation 1.9, the value of z can be corrected as shown in Equation 2.1.

[0164]

[0165] Simplifying equation 2.1, we can get:

[0166]

[0167] Similarly, combined Figure 4c , Figure 4d The spatial relationship shown can also be used to correct the value of z, which will not be elaborated in detail here.

[0168] Therefore, optionally, Figure 3 Before step S302 shown, Figure 3 The method shown also includes step S305 (not shown in the figure):

[0169] S305: Obtain target height.

[0170] Optionally, the target height may be a target height input by the user according to actual conditions when the target object positioning device is actually used after the target object positioning device leaves the factory, for example, an actual installation height of the target object positioning device input by the user.

[0171] Optionally, step S305 may also be located before step S301, which is not limited in this application.

[0172] Based on the above technical solution, in the process of using video and audio for coordinated positioning, when the reference coordinate systems used for video positioning and audio positioning are different, the position information of the target object determined by video positioning and the position information of the target object determined by audio positioning can be used in combination to improve the accuracy of positioning the target object.

[0173] The embodiment of the present application provides a target object positioning device, which is used to execute each step in the above-mentioned target object positioning method. The embodiment of the present application can divide the functional modules of the target object positioning device according to the above-mentioned method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. The division of modules in the embodiment of the present application is schematic, which is only a logical function division. There can be other division methods in actual implementation.

[0174] In the case of dividing each functional module into corresponding functional modules, Figure 8 FIG. 2 shows a possible structural diagram of the target object positioning device involved in the above embodiment. Figure 8 As shown, the target object positioning device 800 includes: an acquisition unit 801 and a processing unit 802 .

[0175] An acquisition unit 801 is used to acquire first information;

[0176] A processing unit 802 is configured to determine first position information of each first object in the at least one first object in a first coordinate system according to the first information;

[0177] The processing unit 802 is further configured to determine the second position information of each first object in the second coordinate system according to the mapping relationship and the first position information of each first object, wherein the mapping relationship is the relationship between the first position information of the object and the second position information of the object, and the first coordinate system is different from the second coordinate system;

[0178] The acquisition unit 801 is further used to acquire second information;

[0179] The processing unit 802 is further configured to determine third position information of each second object in the at least one second object in the second coordinate system according to the second information;

[0180] The processing unit 802 is further configured to determine a target object according to the third position information of each second object and the second position information of each first object to locate the target object, where the target object is an object corresponding to when the third position information matches the second position information;

[0181] The first information is image information, and the second information is audio information; or the first information is audio information, and the second information is image information.

[0182] In one example, the first position information is the first horizontal azimuth of the object in the first coordinate system, and the second position information is the second horizontal azimuth of the object in the second coordinate system; the processing unit is specifically used to determine the second horizontal azimuth of each first object based on the first horizontal azimuth of each first object, z and / or x corresponding to each first object, and y corresponding to each first object; wherein, if the first information is image information and the second information is audio information, x is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system, y is the vertical distance between the first object and the origin of the first coordinate system or the origin of the second coordinate system, and z is the horizontal distance between the first object and the origin of the second coordinate system; if the first information is audio information and the second information is image information, x is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system, y is the vertical distance between the first object and the origin of the first coordinate system or the origin of the second coordinate system, and z is the horizontal distance between the first object and the origin of the first coordinate system.

[0183] In one example, the value of y is determined according to the imaging size of the target area of ​​the object in the acquired image information.

[0184] In one example, the value of y is negatively correlated with the imaging size of the target area of ​​the object in the acquired image information.

[0185] In one example, the value of y and the imaging size of the target area of ​​the object satisfy the following formula: y=am d , where a is a preset constant greater than 0, d is a preset constant less than 0, and m is the imaging size of the target area of ​​the object.

[0186] In one example, z satisfies the following formula:

[0187] In one example, the mapping relationship satisfies the following formula:

[0188]

[0189] Wherein, if the first information is image information and the second information is audio information, α1 is the first horizontal azimuth, and α1 is greater than 90°, and β1 is the second horizontal azimuth. If the first information is audio information and the second information is image information, β1 is the first horizontal azimuth, α1 is the second horizontal azimuth, and α1 is greater than 90°.

[0190] In one example, the mapping relationship satisfies the following formula:

[0191]

[0192] Among them, if the first information is image information and the second information is audio information, then α1 is the first horizontal azimuth angle, and α1 is less than 90°, and β1 is the second horizontal azimuth angle; if the first information is audio information and the second information is image information, then β1 is the first horizontal azimuth angle, α1 is the second horizontal azimuth angle, and α1 is less than 90°.

[0193] In one example, the mapping relationship satisfies the following formula:

[0194]

[0195] Among them, if the first information is image information and the second information is audio information, then α1 is the first horizontal azimuth angle, and α1 is equal to 90°, and β1 is the second horizontal azimuth angle; if the first information is audio information and the second information is image information, then β1 is the first horizontal azimuth angle, α1 is the second horizontal azimuth angle, and α1 is equal to 90°.

[0196] In one example, when the position of the object remains unchanged, the imaging size of the target area of ​​the object is negatively correlated with the target height, and the target height is the distance between the origin of the first coordinate system or the origin of the second coordinate system and the reference surface. Based on this implementation, the imaging size of the target area of ​​the object can be determined according to the actual value of the target height, that is, the height at which the target tracking device is actually installed, which can improve the accuracy of the determined y value and z value, thereby making the mapping relationship more accurate and improving the accuracy of positioning the target object.

[0197] In one example, the target height and y satisfy the following formula: Wherein, y1 is the distance between the object and the origin of the first coordinate system or the origin of the second coordinate system after being mapped to the target vertical plane, and h is the target height.

[0198] In one example, the value of y1 is determined according to the imaging size of the target area of ​​the object in the acquired image information.

[0199] In one example, the value of y1 is negatively correlated with the imaging size of the target area of ​​the object in the acquired image information.

[0200] In one example, the value of y1 and the imaging size of the target area of ​​the object satisfy the following formula: y1 = am d , where a is a preset constant greater than 0, d is a preset constant less than 0, and m is the imaging size of the target area of ​​the object.

[0201] Each module of the above-mentioned target object positioning device can also be used to perform other actions in the above-mentioned method embodiment. All relevant contents of each step involved in the above-mentioned method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0202] Of course, the target object positioning device provided in the embodiment of the present application includes but is not limited to the above-mentioned units. For example, the target object positioning device may also include a storage unit 803 .

[0203] The storage unit 803 may be used to store program codes and data of the target object positioning device.

[0204] In the case of implementing the functions of the above-mentioned integrated modules in the form of hardware, an embodiment of the present application provides another possible structural diagram of the above-mentioned target and object positioning device.

[0205] like Fig. 9 As shown, the target object positioning device 900 includes a processor 901, a memory 902, a bus 903 and a communication interface 904. The processor 901, the memory 902 and the communication interface 904 may be connected via the bus 903.

[0206] The processor 901 is the control center of the communication device, which can be a processor or a general term for multiple processing elements. For example, the processor 901 can be a general-purpose central processing unit (CPU) or other general-purpose processors. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0207] As an embodiment, the processor 901 may include one or more CPUs, such as Fig. 9 CPU 0 and CPU 1 are shown in .

[0208] The memory 902 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0209] As a possible implementation, the memory 902 may exist independently of the processor 901, and the memory 902 may be connected to the processor 901 via a bus 903 to store instructions or program codes. When the processor 901 calls and executes the instructions or program codes stored in the memory 902, the target object positioning method provided in the embodiment of the present application can be implemented.

[0210] In another possible implementation, the memory 902 may also be integrated with the processor 901 .

[0211] The bus 903 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0212] The communication interface 904 is used to connect with other devices through a communication network. The communication network may be Ethernet, wireless access network, wireless local area network (WLAN), etc. The communication interface 904 may include a receiving unit for receiving data and a sending unit for sending data.

[0213] In one possible design, in the device provided in the embodiment of the present application, the communication interface can also be integrated into the processor.

[0214] It should be pointed out that Fig. 9 The structure shown does not constitute a limitation on the target object positioning device 900. Fig. 9 In addition to the components shown, the target object locating device 900 may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0215] Optionally, an embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; the memory is used to store computer program code, and the computer program code includes computer instructions; when the computer instructions are executed by the processor, the processor executes the method introduced in the above embodiment.

[0216] Optionally, an embodiment of the present application further provides a computer program product carrying computer instructions, which, when executed on a computer, enables the computer to execute the method described in the above embodiment.

[0217] Optionally, an embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the method introduced in the above embodiment.

[0218] Optionally, an embodiment of the present application also provides a chip, including at least one processor and at least one interface circuit, the at least one interface circuit is used to perform transceiver functions and send instructions to at least one processor, when the at least one processor executes the instructions, the at least one processor executes the method introduced in the embodiment of the present application.

[0219] Among them, the target object positioning device, electronic device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0220] It will be appreciated by those skilled in the art that in the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center integrated with one or more available media. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0221] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0222] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical or other forms.

[0223] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple devices. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0224] Through the description of the above implementation methods, the technicians in the relevant field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application can be essentially or in other words, the part that makes the contribution can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, hard disk or optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0225] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for locating a target object, characterized in that: The method comprises: Determine first position information of each first object in the at least one first object in the first coordinate system according to the acquired first information; determining the second position information of each first object in a second coordinate system according to a mapping relationship and the first position information of each first object, wherein the mapping relationship is determined according to types of the first position information and the second position information and is used to convert the first position information of the object into the second position information of the object according to the origin positions of the first coordinate system and the second coordinate system, and the first coordinate system is different from the second coordinate system; Determine third position information of each second object in the at least one second object in the second coordinate system according to the acquired second information; Determine a target object according to the third position information of each second object and the second position information of each first object to locate the target object, the target object being an object corresponding to when the third position information matches the second position information; The first information is image information, and the second information is audio information; or the first information is audio information, and the second information is image information.

2. The method according to claim 1, characterized in that The first position information is a first horizontal azimuth of the object in the first coordinate system, and the second position information is a second horizontal azimuth of the object in the second coordinate system; The determining the second position information of each first object in the second coordinate system according to the mapping relationship and the first position information of each first object includes: Determine the second horizontal azimuth angle of each first object according to the first horizontal azimuth angle of each first object, z and / or x corresponding to each first object, and y corresponding to each first object; Wherein, if the first information is image information and the second information is audio information, the x is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system, the y is the vertical distance between the first object and the origin of the first coordinate system or the origin of the second coordinate system, and the z is the horizontal distance between the first object and the origin of the second coordinate system; If the first information is audio information and the second information is image information, x is the horizontal distance between the origin of the first coordinate system and the origin of the second coordinate system, y is the vertical distance between the first object and the origin of the first coordinate system or the origin of the second coordinate system, and z is the horizontal distance between the first object and the origin of the first coordinate system.

3. The method according to claim 2, characterized in that The value of y is determined according to the imaging size of the target area of ​​the object in the acquired image information.

4. The method according to claim 2 or 3, characterized in that: The value of y is negatively correlated with the imaging size of the target area of ​​the object in the acquired image information.

5. The method according to claim 2, characterized in that: The mapping relationship satisfies the following formula: If the first information is image information and the second information is audio information, then α1 is the first horizontal azimuth angle, and α1 is greater than 90°, and β1 is the second horizontal azimuth angle; If the first information is audio information and the second information is image information, then the β1 is the first horizontal azimuth angle, the α1 is the second horizontal azimuth angle, and the α1 is greater than 90°.

6. The method according to claim 2, characterized in that The mapping relationship satisfies the following formula: Wherein, if the first information is image information and the second information is audio information, then the α1 is the first horizontal azimuth angle, and the α1 is less than 90°, and the β1 is the second horizontal azimuth angle; If the first information is audio information and the second information is image information, then the β1 is the first horizontal azimuth angle, the α1 is the second horizontal azimuth angle, and the α1 is less than 90°.

7. The method according to claim 2, characterized in that The mapping relationship satisfies the following formula: If the first information is image information and the second information is audio information, then α1 is the first horizontal azimuth angle, and α1 is equal to 90°, and β1 is the second horizontal azimuth angle; If the first information is audio information and the second information is image information, then the β1 is the first horizontal azimuth angle, the α1 is the second horizontal azimuth angle, and the α1 is equal to 90°.

8. The method according to claim 2 or 3, characterized in that: When the position of the object remains unchanged, the imaging size of the target area of ​​the object in the acquired image information is negatively correlated with the target height, and the target height is the distance between the origin of the first coordinate system or the origin of the second coordinate system and the reference surface.

9. The method according to claim 8, characterized in that The target height and y satisfy the following formula: Wherein, y1 is the distance between the object and the origin of the first coordinate system or the origin of the second coordinate system after being mapped to the target vertical plane, and h is the target height.

10. The method according to claim 9, characterized in that The value of y1 is determined according to the imaging size of the target area of ​​the object in the acquired image information.

11. A target object positioning device, characterized in that: including an acquisition unit and a processing unit; The acquiring unit is used to acquire first information; The processing unit is used to determine first position information of each first object in the at least one first object in a first coordinate system according to the first information; The processing unit is further used to determine the second position information of each first object in a second coordinate system according to a mapping relationship and the first position information of each first object, wherein the mapping relationship is determined according to types of the first position information and the second position information, and is used to convert the first position information of the object into the second position information of the object according to the origin positions of the first coordinate system and the second coordinate system, and the first coordinate system is different from the second coordinate system; The acquisition unit is further used to acquire second information; The processing unit is further used to determine third position information of each second object in at least one second object in the second coordinate system according to the second information; The processing unit is further configured to determine a target object according to the third position information of each second object and the second position information of each first object to locate the target object, wherein the target object is an object corresponding to when the third position information matches the second position information; The first information is image information, and the second information is audio information; or the first information is audio information, and the second information is image information.

12. An electronic device, characterized in that: It comprises a memory and a processor, the memory is coupled to the processor; the memory is used to store computer program code, the computer program code comprises computer instructions; when the computer instructions are executed by the processor, the processor executes the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on a computing device, cause the computing device to perform the method according to any one of claims 1 to 10.

Images