Positioning method and device based on earphone camera module, earphone and storage medium

Abstract

The invention relates to the field of positioning navigation, in particular to a positioning method and device based on an earphone camera module, an earphone and a storage medium. The method comprises the following steps: controlling the camera module to acquire an image, and identifying a target object in the image; and determining the position of the target object in the image, and determining the position of a wearer of the earphone based on the position of the target object in the image and the position of the target object in a map.

Description

Technical Field

[0001] The present disclosure relates to the field of positioning and navigation, and more specifically, to a positioning method, device, headset and storage medium based on a headset camera module. Background Art

[0002] With the rapid development of science and technology, smart wearable devices have become an indispensable part of people's daily lives. As one of the most common smart wearable devices, the functions of headphones are no longer limited to audio playback and calls. More and more innovative technologies are integrated into headphones to meet the diverse needs of users. Currently, the positioning function is mainly implemented through devices such as smartphones and smart watches. Although these devices are powerful, they may have inaccurate positioning problems in certain scenarios, such as indoors or in museums, exhibition halls and other scenes with poor signals. Summary of the invention

[0003] One object of the present disclosure is to provide a positioning method based on an earphone camera module, which can realize the positioning function through the camera module of the earphone to facilitate subsequent navigation.

[0004] According to a first aspect of the present disclosure, a positioning method based on an earphone camera module is provided, wherein the earphone is provided with a camera module, and the method comprises:

[0005] Controlling the camera module to acquire an image and identifying a target object in the image;

[0006] The position of the target object in the image is determined, and the position of the wearer of the headset is determined based on the position of the target object in the image and the position of the target object in the map.

[0007] Optionally, the headset is provided with a magnetometer, and the magnetometer is used to determine the front direction of the headset wearer; before determining the position of the target object in the image, the method includes:

[0008] In the width direction, the image is divided according to a preset number of pixels, so as to generate a plurality of image areas of equal or unequal widths, wherein each image area corresponds to an angle value, and the angle value represents the relative angle between the front direction of the wearer and the target object;

[0009] Optionally, determining the position of the target object in the image, and determining the position of the wearer of the headset based on the position of the target object in the image and the position of the target object in the map, includes:

[0010] Determine a target image region where a middle pixel of the target object in a width direction is located;

[0011] Determine the relative angle between the wearer's front direction and the target object according to the target image area;

[0012] Determining a distance between the target object and the headset wearer;

[0013] Based on the relative angle, the distance and the position of the target object in the map, the position of the headphone wearer in the map is determined.

[0014] Optionally, determining the distance between the target object and the headphone wearer includes:

[0015] Determine the number of pixels of the target object in at least one of the length direction and the width direction and the camera parameters when the camera module captures the image;

[0016] The distance between the target object and the headphone wearer is determined according to the number of pixels, the camera parameters and a pre-stored first corresponding relationship, wherein the first corresponding relationship includes a corresponding relationship between the distance between the target object and the wearer and the number of pixels in the length and width directions under the corresponding camera parameters.

[0017] Optionally, the headset includes a first headset body and a second headset body, and the first headset body and the second headset body are respectively provided with a camera module; before determining the position of the target object in the image, the method includes:

[0018] In the width direction, the first image is divided according to a preset number of pixels to generate a plurality of image areas of equal width or unequal width, wherein the first image is an image acquired by the camera module of the first headset body, and each image area corresponds to an angle value, and the angle value represents the relative angle between the front direction of the wearer and the target object;

[0019] In the width direction, the second image is divided according to a preset number of pixels to generate multiple image areas of equal or unequal width, wherein the second image is an image acquired by the camera module of the second earphone body, and each image area corresponds to an angle value, which represents the relative angle between the front direction of the wearer and the target object.

[0020] Optionally, the target object includes a first target object and a second target object, and determining the position of the wearer of the headset based on the position of the target object in the image and the position of the target object in the map includes:

[0021] Determine a first middle pixel of the first target object in a width direction, and determine a first image area of ​​the first middle pixel on the first image or the second image;

[0022] Determine a second middle pixel of the second target object in the width direction, and determine a second image area of ​​the second middle pixel on the first image or the second image;

[0023] determining a first angle between the first target object and the front direction of the wearer based on the first image area;

[0024] determining a second angle between the second target object and the front direction of the wearer based on the second image area;

[0025] The position of the headphone wearer is determined according to the positions of the first target object and the second target object in the map and the first angle and the second angle.

[0026] Optionally, after determining the position of the wearer of the headset, the method further comprises:

[0027] A route to a destination is determined based on the position of the wearer, and a voice message is played to the user through the headset, wherein the voice message is used to guide the wearer to move along the route to the destination.

[0028] According to a second aspect of the present disclosure, a positioning device based on an earphone camera module is provided, wherein the earphone is provided with a camera module, and the device comprises:

[0029] A control module, used to control the camera module to acquire an image and identify a target object in the image;

[0030] The determination module is used to determine the position of the target object in the image, and determine the position of the wearer of the headset based on the position of the target object in the image.

[0031] According to a third aspect of the present disclosure, there is provided a headset, comprising a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the steps of any one of the methods described in the first aspect are implemented.

[0032] According to a fourth aspect of the present disclosure, there is provided a storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0033] One technical effect of the present disclosure is that a positioning method based on a headset camera module is provided. Specifically, an image can be acquired through the camera module, and a target object in the image can be identified, and then the specific position of the headset wearer can be determined according to the position of the target object in the image and the map. In this way, the user can realize self-positioning through the camera module in the headset, especially in scenes with poor signals such as indoors, and can realize self-positioning in this way for subsequent navigation or other services that require location.

[0034] Other features and advantages of the embodiments of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments of the present disclosure.

[0036] Figure 1 is a flow chart of a positioning method based on an earphone camera module according to an embodiment;

[0037] Figure 2 is a schematic diagram of dividing an image region according to an embodiment;

[0038] Figure 3 is a schematic diagram of a map according to one embodiment;

[0039] Figure 4 is a schematic diagram of a map according to one embodiment;

[0040] Figure 5 is a schematic diagram of a map according to one embodiment;

[0041] Figure 6 It is a structural schematic diagram of a positioning device based on an earphone camera module according to an embodiment. DETAILED DESCRIPTION

[0042] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0043] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0044] Techniques and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the techniques and equipment should be considered part of the specification.

[0045] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0046] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0047] The embodiment of the present application provides a method for positioning a headset camera module, which can be applied to a headset or an electronic device connected to the headset. The headset is provided with a camera module, and the camera module can capture images or videos under control. The method includes steps S11-S12.

[0048] Step S11, controlling the camera module to acquire an image and identifying a target object in the image.

[0049] In this example, the earphones can be TWS (True Wireless Stereo, true wireless stereo earphones) earphones, or they can be headphones or open earphones. Taking TWS earphones as an example, it can include two earphones suitable for the left and right ears of the user, and a charging storage device. In an embodiment of the present application, the camera module of the earphone can be set on both earphone bodies, and the processing device can be set on the earphone body or in the charging storage device. In the headphone system, the camera module can be set in a group near the left and right ears, respectively, to detect heart rate data respectively.

[0050] In the embodiment of the present application, the earphone body can be a small earplug or a comfortable ear hook or ear clip worn on the left and right sides of a wireless earphone; at the same time, it can also be a large earmuff in a head-mounted headset that wraps around the ears and provides excellent sound insulation, or the main part of an all-in-one earphone that integrates multiple functions and also covers the ears.

[0051] In this embodiment, the camera module of the headset can be controlled to start shooting, and then the target object in the image can be identified by an image recognition algorithm. The image recognition algorithm can be a pre-trained YOLO algorithm or an SSD algorithm, and the target object can be a pre-stored or set marker. For example, in a museum scene, the target object can be a cultural relic, signboard or other object exhibited in the venue. In an embodiment of the present application, map data such as warehouses, museums, schools, etc. can be pre-stored, and the location of the target object in the map can also be pre-stored. In an embodiment of the present application, the number and type of target objects can be pre-set according to actual needs, and the relevant information of the target object can also be pre-stored accordingly.

[0052] Step S12, determining the position of the target object in the image, and determining the position of the wearer of the headset based on the position of the target object in the image and the position of the target object in the map.

[0053] In an embodiment of the present application, after obtaining the image taken by the headset and identifying the target object in the image, the position of the headset wearer relative to the target object can be determined based on the position of the target object in the image. At the same time, based on the position of the target object in the map, the location of the headset wearer in the map can also be further determined to facilitate subsequent navigation and other services.

[0054] In this example, a positioning method based on a headset camera module is provided. Specifically, an image can be acquired through the camera module, and a target object in the image can be identified. Then, the specific position of the headset wearer can be determined based on the position of the target object in the image and the map. In this way, the user can realize self-positioning through the camera module in the headset, especially in scenes with poor signals such as indoors. In this way, self-positioning can be realized for subsequent navigation or other services that require location.

[0055] In an example of this embodiment, a magnetometer is provided on the headset, and the magnetometer is used to determine the front direction of the headset wearer; before determining the position of the target object in the image, the method includes: dividing the image in the width direction according to a preset number of pixels, thereby generating a plurality of image areas of equal or unequal widths, wherein each image area corresponds to an angle value, and the angle value represents the relative angle between the front direction of the wearer and the target object; determining the position of the target object in the image includes: determining the target image area where the middle pixel of the target object in the width direction is located.

[0056] In this example, the headset may be provided with a magnetometer, which can determine the direction in which the wearer is facing when the headset is worn, such as due east, 30 degrees north of due east, etc., and the specific angle based on the earth's magnetic field. In this embodiment, the image captured by the camera module can be divided into multiple areas in the width direction. The preset number of pixels can be set according to the parameters of the camera module and the actual algorithm requirements. In this example, the image area can be as follows: Figure 2 As shown, a dividing line is set every preset number of pixels to divide the image area. The preset number of pixels may include multiple ones. Since the image captured by the camera module may be distorted, a larger preset number of pixels may be used on both sides of the image, and a smaller preset number of pixels may be used in the center of the image, so that the intervals of the angle values ​​corresponding to each divided image area are basically the same. In one example, among these image areas, there may also be an image area representing the front direction of the user.

[0057] In this embodiment, the headset can be tested and stored in advance with a preset number of pixels when it leaves the factory, that is, the correspondence between the image area and the angle value. Specifically, during the test, the user can wear the headset and mark the shooting angle, and then divide the image with the angle mark taken to determine the angle value directly in front of the user corresponding to the image area divided by the preset number of pixels.

[0058] In an example of this embodiment, determining the position of the target object in the image, and determining the position of the wearer of the headset based on the position of the target object in the image and the position of the target object in the map, includes: determining the target image area where the middle pixel of the target object in the width direction is located; determining the relative angle between the wearer's front direction and the target object according to the target image area; determining the distance between the target object and the wearer of the headset; and determining the position of the wearer of the headset in the map based on the relative angle, the distance, and the position of the target object in the map.

[0059] After acquiring the image actually taken by the camera, the image is divided based on the preset number of pixels. After the division, the target image area where the middle pixel of the target object in the width direction is located is determined. The middle pixel of the target object in the width direction, that is, the middle pixel in the horizontal or width direction, for example, the target object occupies a total of 100 columns of pixels in the width direction, then the middle pixel can be the pixel of the target object in the 50th or 51st column; if the target object occupies 99 columns of pixels in the width direction, then the pixel in the 50th column can be used as the middle pixel. After determining the middle pixel, it is possible to further determine which image area the middle pixel is in, so that the angle value corresponding to the image area can be used as the relative angle between the target object and the front direction of the wearer. Figure 3 As shown, Figure 3 This is a simple map example. The top of the map is north, the black dot is the position of the wearer, and the arrow is the direction in front of the wearer. The direction in front of the wearer can be determined by a magnetometer, which is due north in this example. Based on this, after determining the angle between the target object and the front of the wearer, it is only necessary to further determine the distance between the target object and the wearer to accurately determine the position and direction of the wearer in the map. In this embodiment, the distance between the two can be determined based on other ranging sensors or a camera between the two headset bodies combined with a binocular vision algorithm.

[0060] In this example, the position of the headphone wearer in the map can be determined based on the relative angle, distance and position of the target object and the wearer, thereby solving the problem of difficult and inaccurate indoor positioning.

[0061] In an example of the present embodiment, determining the distance between the target object and the headphone wearer includes: determining the number of pixels of the target object in at least one direction of the length direction and the width direction and the camera parameters when the camera module captures the image; determining the distance between the target object and the headphone wearer based on the number of pixels, the camera parameters and a pre-stored first correspondence, the first correspondence including the correspondence between the distance between the target object and the wearer and the number of pixels in the length and width directions under the corresponding camera parameters.

[0062] In this embodiment, the distance between the target object and the headset camera module and the number of pixels of the target object in the length and width directions, as well as the corresponding relationship between the camera parameters can be tested and stored in advance. The camera parameters can be focal length, field of view angle parameters, etc. In the test scenario, the distance and corresponding number of pixels of each target object under each camera parameter can be tested in advance to obtain the corresponding relationship. In the actual scenario, due to the angle problem, the image of the target object may be deformed in one direction in the length direction or the width direction. Therefore, the number of pixels of the target object in a certain direction can be selected according to the actual situation, and based on the corresponding preset relationship, the distance between the target object and the wearer can be determined.

[0063] In an example of the present embodiment, the earphone includes a first earphone body and a second earphone body, and the first earphone body and the second earphone body are respectively provided with a camera module; before determining the position of the target object in the image, the method includes: in the width direction, dividing the first image according to a preset number of pixels, so as to generate a plurality of image areas of equal width or unequal width, wherein the first image is an image acquired by the camera module of the first earphone body, and each image area corresponds to an angle value, and the angle value represents the relative angle between the front direction of the wearer and the target object; in the width direction, dividing the second image according to a preset number of pixels, so as to generate a plurality of image areas of equal width or unequal width, wherein the second image is an image acquired by the camera module of the second earphone body, and each image area corresponds to an angle value, and the angle value represents the relative angle between the front direction of the wearer and the target object.

[0064] In this example, the two earphone bodies in the earphone can be respectively provided with camera modules. In this example, the two earphone bodies are earphone bodies worn on the left ear and the right ear of the user respectively. When positioning, the camera modules on the two earphone bodies can be used to take pictures at the same time. In this embodiment, the image taken by each earphone body can be divided into image areas respectively.

[0065] In an example of the present embodiment, the target object includes a first target object and a second target object, and the position of the wearer of the headset is determined based on the position of the target object in the image and the position of the target object in the map, including: determining a first intermediate pixel of the first target object in the width direction, and determining a first image area of ​​the first intermediate pixel on the first image or the second image; determining a second intermediate pixel of the second target object in the width direction, and determining a second image area of ​​the second intermediate pixel on the first image or the second image; determining a first angle between the first target object and the front direction of the wearer based on the first image area; determining a second angle between the second target object and the front direction of the wearer based on the second image area; and determining the position of the wearer of the headset according to the positions of the first target object and the second target object in the map and the first angle and the second angle.

[0066] In this embodiment, when multiple target objects appear at the same time, the current position of the wearer can be determined by determining the relative angle between any two target objects and the front of the wearer. In this example, the image area where each target object is located can be determined first, which can be determined by determining the image area where the first middle pixel of each target object in the width direction is located.

[0067] When the first target object and the second target object are both photographed by the camera module of one of the two headset bodies, that is, Figure 4 In the situation shown, at this time, the angles of the first target object and the second target object relative to the front of the wearer can be determined respectively, for example, the first target object is in a direction of 30 degrees to the left in front, and the second target object is in a direction of 60 degrees to the left in front. Since the positions of the first target object and the second target object in the map are pre-stored, at this time, the position of the headphone wearer at the intersection of the dotted lines in the map can be determined based on the positions of the first target object and the second target object on the map and these two angles.

[0068] In another example, if Figure 5 As shown, the first target object and the second target object are respectively photographed by the left and right earphone bodies. Similar to the previous example, the angle between the first target object and the front of the wearer, such as 30 degrees in front of the left side, and the angle between the second target object and the front of the wearer, such as 30 degrees in front of the right side, can be determined respectively. Based on the positions of the first target object and the second target object, the position of the wearer, that is, the position of the intersection of the dotted lines in the figure, can be determined.

[0069] In this example, in order to further improve the positioning accuracy of the wearer, a magnetometer can also be provided in the headset to determine the specific direction in front of the user and to calibrate the user's position.

[0070] In this embodiment, the above steps can be repeatedly executed at a preset frequency, and images can be continuously acquired through the camera module of the headset to continuously update the user's own location.

[0071] In an example of this embodiment, after determining the position of the wearer of the headset, the method further includes: determining a route to the destination based on the wearer's position, and playing a voice message to the user through the headset, wherein the voice message is used to guide the wearer to move along the route to the destination.

[0072] In this example, after the wearer's location is determined, a route from the wearer's current location to the destination location can be constructed in response to the location of the destination input in advance or afterwards by the wearer to achieve indoor navigation. In this example, the location of the destination can be the location of a certain target object, such as the location of an exhibit in a museum, the location of an elevator, the location of a toilet, etc., or it can be a location input by the user. In this example, the headset can also prompt the user with the running route to the destination by playing a voice message. For example, "Walk forward 50 meters and turn right to reach the destination."

[0073] like Figure 6 As shown, an embodiment of the present application also provides a positioning device 100 based on a headset camera module. The headset is provided with a camera module, and the device includes: a control module 101, used to control the camera module to acquire an image and identify a target object in the image; a determination module 102, used to determine the position of the target object in the image, and determine the position of the wearer of the headset based on the position of the target object in the image.

[0074] Optionally, the earphone is provided with a magnetometer, and the magnetometer is used to determine the forward direction of the earphone wearer; the device comprises:

[0075] A division module, used for dividing the image in the width direction according to a preset number of pixels, so as to generate a plurality of image regions of equal or unequal widths, wherein each image region corresponds to an angle value, and the angle value represents the relative angle between the front direction of the wearer and the target object;

[0076] Optionally, the determination module is specifically used to include:

[0077] Determine the target image region where the middle pixel of the target object in the width direction is located;

[0078] Determine the relative angle between the wearer's front direction and the target object according to the target image area;

[0079] Determine the distance of the target object from the headset wearer;

[0080] The headset wearer's position in the map is determined based on the relative angle, distance, and position of the target object in the map.

[0081] Optionally, determining a distance between the target object and the headset wearer includes:

[0082] Determine the number of pixels of the target object in at least one of the length direction and the width direction and the camera parameters when the camera module captures the image;

[0083] The distance between the target object and the headphone wearer is determined according to the number of pixels, the camera parameters and a pre-stored first corresponding relationship, wherein the first corresponding relationship includes a corresponding relationship between the distance between the target object and the wearer and the number of pixels in the length and width directions under the corresponding camera parameters.

[0084] Optionally, the earphone includes a first earphone body and a second earphone body, and the first earphone body and the second earphone body are respectively provided with a camera module; the device further includes:

[0085] A division module is used to divide the first image in the width direction according to a preset number of pixels, so as to generate a plurality of image areas of equal or unequal widths, wherein the first image is an image acquired by the camera module of the first earphone body, and each image area corresponds to an angle value, which represents the relative angle between the front direction of the wearer and the target object; in the width direction, the second image is divided according to a preset number of pixels, so as to generate a plurality of image areas of equal or unequal widths, wherein the second image is an image acquired by the camera module of the second earphone body, and each image area corresponds to an angle value, which represents the relative angle between the front direction of the wearer and the target object.

[0086] Optionally, the determination module is specifically used to include:

[0087] Determine a first middle pixel of the first target object in the width direction, and determine a first image area of ​​the first middle pixel on the first image or the second image;

[0088] Determine a second middle pixel of the second target object in the width direction, and determine a second image area of ​​the second middle pixel on the first image or the second image;

[0089] Determine a first angle between the first target object and the front direction of the wearer based on the first image area;

[0090] determining a second angle between the second target object and the front direction of the wearer based on the second image area;

[0091] The position of the headphone wearer is determined according to the positions of the first target object and the second target object in the map and the first angle and the second angle.

[0092] Optionally, the device also includes a navigation module for determining a route to a destination based on the wearer's location and playing a voice message to the user through headphones, wherein the voice message is used to guide the wearer to move along the route to the destination.

[0093] An embodiment of the present application also provides an earphone, including a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, any one of the above-mentioned positioning method embodiments based on the earphone camera module is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0094] An embodiment of the present application also provides a storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, any one of the above-mentioned positioning method embodiments based on the earphone camera module is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0095] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0096] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0097] The embodiments of the present disclosure may be systems, methods and / or computer program products. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the embodiments of the present disclosure.

[0098] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0099] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0100] The computer program instructions for performing the operation of the embodiments of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of a computer-readable program instruction, and the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the embodiments of the present disclosure.

[0101] Various aspects of the embodiments of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0102] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0103] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0104] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and a part of a module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0105] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A positioning method based on an earphone camera module, characterized in that: The headset is provided with a camera module, and the method comprises: Controlling the camera module to acquire an image and identifying a target object in the image; The position of the target object in the image is determined, and the position of the wearer of the headset is determined based on the position of the target object in the image and the position of the target object in the map.

2. The method according to claim 1, characterized in that The earphone is provided with a magnetometer, and the magnetometer is used to determine the front direction of the earphone wearer; Before determining the position of the target object in the image, the method includes: In the width direction, the image is divided according to a preset number of pixels to generate a plurality of image areas of equal or unequal width, wherein each image area corresponds to an angle value, and the angle value represents the relative angle between the front direction of the wearer and the target object.

3. The method according to claim 2, characterized in that The determining the position of the target object in the image, and determining the position of the wearer of the headset based on the position of the target object in the image and the position of the target object in the map, includes: Determine a target image region where a middle pixel of the target object in a width direction is located; Determine the relative angle between the wearer's front direction and the target object according to the target image area; Determining a distance between the target object and the headset wearer; Based on the relative angle, the distance and the position of the target object in the map, the position of the headphone wearer in the map is determined.

4. The method according to claim 3, characterized in that: Determining the distance between the target object and the headphone wearer includes: Determine the number of pixels of the target object in at least one of the length direction and the width direction and the camera parameters when the camera module captures the image; The distance between the target object and the headphone wearer is determined according to the number of pixels, the camera parameters and a pre-stored first corresponding relationship, wherein the first corresponding relationship includes a corresponding relationship between the distance between the target object and the wearer and the number of pixels in the length and width directions under the corresponding camera parameters.

5. The method according to claim 3, characterized in that: The earphone comprises a first earphone body and a second earphone body, wherein the first earphone body and the second earphone body are respectively provided with a camera module; Before determining the position of the target object in the image, the method includes: In the width direction, the first image is divided according to a preset number of pixels to generate a plurality of image areas of equal width or unequal width, wherein the first image is an image acquired by the camera module of the first headset body, and each image area corresponds to an angle value, and the angle value represents the relative angle between the front direction of the wearer and the target object; In the width direction, the second image is divided according to a preset number of pixels to generate multiple image areas of equal or unequal width, wherein the second image is an image acquired by the camera module of the second earphone body, and each image area corresponds to an angle value, which represents the relative angle between the front direction of the wearer and the target object.

6. The method according to claim 5, characterized in that The target object includes a first target object and a second target object, and determining the position of the wearer of the headset based on the position of the target object in the image and the position of the target object in the map includes: Determine a first middle pixel of the first target object in a width direction, and determine a first image area of ​​the first middle pixel on the first image or the second image; Determine a second middle pixel of the second target object in the width direction, and determine a second image area of ​​the second middle pixel on the first image or the second image; determining a first angle between the first target object and the front direction of the wearer based on the first image area; determining a second angle between the second target object and the front direction of the wearer based on the second image area; The position of the headphone wearer is determined according to the positions of the first target object and the second target object in the map and the first angle and the second angle.

7. The method according to claim 1, characterized in that After determining the position of the wearer of the headset, the method further comprises: A route to a destination is determined based on the position of the wearer, and a voice message is played to the user through the headset, wherein the voice message is used to guide the wearer to move along the route to the destination.

8. A positioning device based on an earphone camera module, characterized in that: The headset is provided with a camera module, and the device comprises: A control module, used to control the camera module to acquire an image and identify a target object in the image; The determination module is used to determine the position of the target object in the image, and determine the position of the wearer of the headset based on the position of the target object in the image.

9. A headset, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images