Position node tracking

Abstract

A system and method for tracking location nodes of one or more mobile objects, the tracking including scanning for one or more mobile objects, detecting one or more mobile objects, creating a temporary location node around one or more detected mobile objects, searching a node pool for existing location nodes in the vicinity of the temporary location node, setting a timer for the existing location nodes in the vicinity of the temporary location node, the timer having a predetermined time limit, and setting a timer for temporary location nodes not in the vicinity of an existing location node, creating a new location node added to the node pool and setting a timer associated with the new location node, the timer associated with the new location node having a predetermined time limit.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to position node tracking using a dynamic vision sensor (DVS). BACKGROUND

[0002] The sweet spot of a high fidelity audio system is the ideal listening position and is an important factor in ensuring that the listener gets the best sound quality in the listening environment. In a traditional high fidelity audio system, a technician will determine the sweet spot and configure the audio system according to the user's requirements. Once this setup is complete, the sweet spot will remain fixed.

[0003] Beamforming speakers introduce an improvement that allows the user to adjust the sweet spot according to their desired configuration. Typically, this is done in a system with a pair of beamforming speakers that are able to communicate with an application software on a device such as a cell phone, tablet, laptop, etc. By configuring a graphical menu that maps the relative positions of the two speakers on the application, the user can set a preferred listening position in the listening environment. The speakers will adjust the sound beam and adjust the intensity of the sound beam to the desired listening position. However, the graphical menu does not include information about the actual listening environment. During the first installation of the high fidelity audio system, the relative position size of the two speakers is unknown. Typically, the technician installing the audio system will determine the sweet spot and measure the distance between the left and right speakers. The technician will input this measured distance as a baseline parameter into the user's mobile application. Once the measurement is complete and input, the user can adjust the sweet spot through their mobile application. In theory, the user should be able to adjust the sound to a position that the user prefers to listen to by dragging an icon representing the sweet spot within a region representing the listening environment on the display of the mobile device.

[0004] This approach has some drawbacks. Most users are unsure whether the position they are standing in really matches the sweet spot displayed on the application because the application software is not able to track the user's position. In practice, the user has to use trial and error to match the configuration menu to the actual environment. Furthermore, if the position of the speakers changes, the baseline parameter also changes and a technician has to be called to repeat the installation process. SUMMARY

[0005] A system and method for tracking location nodes of one or more mobile objects, the tracking including scanning for one or more mobile objects, detecting one or more mobile objects, creating a temporary location node around one or more detected mobile objects, searching a pool of nodes for existing location nodes in the vicinity of the temporary location node, setting a timer for the existing location nodes in the vicinity of the temporary location node, the timer having a predetermined time limit, and setting a timer for temporary location nodes not in the vicinity of an existing location node, creating a new location node added to the pool of nodes and setting a timer associated with the new location node, the timer associated with the new location node having a predetermined time limit.

[0006] In another example, when none of the objects in an existing location node move before the timer associated with the existing location node expires, the existing location node is deleted from the pool of nodes.

[0007] In another example, when at least one of the objects in an existing location node moves before the timer associated with the existing location node expires, the timer associated with the existing location node is reset and the existing location node remains in the pool of nodes.

[0008] In another example, when at least one of the objects in an existing location node moves outside of the existing location node but remains in the vicinity of the existing location node, a new location node is created to replace the existing location node and a timer associated with the new location node is set, the existing location node is deleted from the pool of nodes.

[0009] In another example, when at least one of the objects in an existing location node moves to a location outside of the existing location node, a new location node is created to replace the existing location node, a timer associated with the new location node is set, and the existing location node remains in the pool of nodes for the duration of the timer associated with the existing location node.

[0010] In another example, when there are two or more existing location nodes in the pool of nodes and one or more objects are in close proximity to each other, the two or more existing location nodes are merged into one location node, a timer associated with the one location node will be reset, and the other existing location nodes will be deleted from the pool of nodes. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is an example electronic device that can include one or more aspects of an automatic beam forming (ABF) system;

[0012] Figure 2 is an example of a listening environment;

[0013] Figure 3 is a block diagram of an example application of an ABF system;

[0014] Figure 4 is a flowchart of a general method of automatic beamforming;

[0015] Figure 5 is an example of a scan-and-lock mode;

[0016] Figure 6 is an example of a scan-and-follow mode;

[0017] Figure 7 is a flowchart of a method for position node tracking;

[0018] Figure 8 rules for position node tracking are shown;

[0019] Figure 9 rules for position node tracking are shown;

[0020] Figure 10 rules for position node tracking are shown;

[0021] Figure 11 rules for position node tracking are shown;

[0022] Figure 12 is a block diagram of a speaker arrangement and target positions;

[0023] Figure 13 is a perspective view of an LED ring;

[0024] Figure 14A is an example of a detected LED pattern;

[0025] Figure 14B is an example of a detected LED pattern;

[0026] Figure 15A is an example of a detected LED pattern;

[0027] Figure 15B is an example of a detected LED pattern; and

[0028] Figure 16 is a flowchart of a method for automatic calibration.

[0029] The elements and steps in the drawings are presented for simplicity and clarity and are not necessarily presented in any particular order. For example, steps that are performed concurrently or in different orders are sometimes shown as consecutive steps to improve the flow of the disclosure. DETAILED DESCRIPTION

[0030] While various aspects of the present disclosure are described with reference to a beamforming speaker system in a listening environment, the present disclosure is not limited to such embodiments, and additional modifications, applications, and functions can be implemented without departing from the scope of the present disclosure. In the drawings, like reference numerals will be used to illustrate like parts. Those skilled in the art will recognize that various components set forth herein can be altered without departing from the scope of the present disclosure.

[0031] Figure 1 is a block diagram of an example electronic device 100 that can include one or more aspects of an example location node tracking system. The electronic device 100 can include a set of instructions that can be executed to cause the electronic device 100 to perform one or more methods and computer-based functions, such as detecting a user, creating one or more location nodes associated with the user, tracking one or more location nodes in a node pool, detecting a gesture, calculating an angle and a distance between a speaker and the user, configuring an optimal listening position, adjusting a sound beam according to the calculated angle, and adjusting a sound beam intensity and a propagation delay according to the calculated distance. The electronic device 100 can operate as a standalone device, can be included as a functional component in another device, or can be connected, e.g., using a network, to other computer systems, devices, or peripheral devices.

[0032] In an example of a networked deployment, the electronic device 100 can operate as a server in a server-client user network environment, or as a client user computer, as a peer computer system in a peer-to-peer (or distributed) network environment, or in other various ways. The electronic device 100 can also be implemented as or incorporated into a variety of electronic devices, such as a desktop computer and a laptop computer, a handheld device such as a smartphone and a tablet computer, a portable media device such as a recording, playing, and gaming device, a home appliance, an office equipment, a set-top box, an automotive electronic device such as a head unit and a navigation system, or any other machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. The electronic device 100 can be implemented using electronic devices that provide voice, audio, video, and / or data communication. While a single electronic device 100 is illustrated, the term “device” can include a collection of devices or sub-devices that individually or collectively execute one or more sets of instructions to perform one or more electronic functions of an ABF system, as described in detail below.

[0033] The electronic device 100 can include a processor 102, such as a central processing unit (CPU), a graphics processing unit (GPU), or both. The processor 102 can be a component in a variety of systems. For example, the processor 102 can be part of a beam-adjusting speaker. Also, the processor 102 can include one or more general processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays, servers, networks, digital circuits, analog circuits, combinations thereof, or other now known or later developed devices for analyzing and processing data. The processor 102 can implement software programs, such as code written in any suitable programming language for the intended functions of the electronic device 100.

[0034] The electronic device 100 can include a memory, such as a memory 104 that can communicate via a bus 106. The memory 104 can be or include a main memory, static memory, or dynamic memory. The memory 104 can include a non-transitory storage device. The memory 104 can also include a computer-readable storage medium, such as various types of volatile and non-volatile storage media including random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, magnetic tape or disk, optical media, and the like. Also, the memory can include a non-transitory, tangible medium that stores software. The software can be stored as an image or other format in electronic form (such as by optical scanning), then compiled, interpreted, or otherwise processed.

[0035] In one example, the memory 104 includes a cache or random access memory for the processor 102. In alternative examples, the memory 104 can be separate from the processor 102, such as a cache memory or processor, system memory, or other memory. The memory 104 can be or include an external storage device or database for storing data. Examples include a hard disk drive, compact disk, digital video disk, universal serial bus, memory stick, floppy disk, or other device for storing data. For example, the electronic device 100 can include a computer-readable medium 108 in which one or more sets of software or instructions can be embedded. The processor 102 and the memory 104 can also include a non-transitory computer-readable storage medium having instructions or software.

[0036] The memory 104 can be used to store instructions that can be executed by the processor 102. The functions, acts, or tasks illustrated in the figures or described can be performed by programmed processor 102 executing instructions stored in memory 104. The functions, acts, or tasks can be independent of the type of instructions set, storage media, processor, or processing strategy and can be performed by software, hardware, integrated circuits, firm ware, micro-code, and the like, working individually or in combination. Similarly, the processing strategy can include multiprocessing, multitasking, parallel processing, and the like.

[0037] The instructions can include one or more methods described herein, including aspects of the electronic device 100 and / or the ABF system 122. During execution by the electronic device 100, the instructions 110 can reside, in whole or in part, within the memory 104 or the processor 102.

[0038] The electronic device 100 can include a non-transitory computer-readable medium that includes the instructions 110 or receives and executes the instructions 110 in response to propagated signals, so that a device connected to the network 112 can communicate voice, video, audio, images, or other data over the network 112. The instructions 110 can be sent or received over the network 112 via the communication port or interface 114 or using the bus 106. The communication port or interface 114 can be part of the processor 102 or can be a separate component. The communication port or interface 114 can be created in software or can be a physical connection in hardware. The communication port or interface 114 can be configured to connect with a network 112, external media, the one or more speakers 116, the one or more cameras 118, the one or more sensors 120, or other components of the electronic device 100, or combinations thereof. The connection with the network 112 can be a physical connection, such as a wired Ethernet connection or can be established wirelessly. The additional connections with other components of the electronic device 100 can also be physical connections or can be established wirelessly. The network 112 can alternatively be directly connected to the bus 106.

[0039] The electronic device can include one or more speakers 116 installed in a vehicle, a living space, or a venue, such as beamforming speakers. The speakers 116 can be part of a stereo surround sound system and the ABF system 122.

[0040] To perform the functions of the ABF system 122, the processor 102 or other components can manipulate or process sound signals sent to the speakers 116. In particular, when the speakers 116 include beamforming speakers, a sound signal can be sent to each speaker in a speaker pair. The signals can be processed individually or jointly. The electronic device 100 can include instructions for adjusting the phase, amplitude, and / or delay of each sound signal delivered to the speakers 116. The phase, amplitude, and / or delay can be controlled in a manner that produces a desired coverage pattern.

[0041] The electronic device 100 can also include one or more sensors 120. The one or more sensors can include one or more proximity sensors, motion sensors, cameras, and dynamic vision sensors (DVS).

[0042] Figure 2One example of a listening environment 200 in which the electronic device can operate is shown. In the illustrated example, the primary speaker 202 has a first detection device 206 that includes a sensor 120. The slave speaker 204 has a second detection device 208 that includes a sensor 120. The primary speaker 202 and the slave speaker 204 can be left and right beamforming speakers of a speaker pair controlled by the electronic device 100 to adjust the sound beam to any angle of 360 degrees, thereby creating a surround sound system. The first detection device 206 and the second detection device 208 have sensors 120 capable of detecting objects and gestures in the listening environment. The sensors 120 can be any of motion sensors, thermal sensors, vision sensors, and cameras but are not limited to these.

[0043] Figure 3 is shown in Figure 2 A diagram showing the variables and necessary parameters of the ABF system within the listening environment shown. The rectangle 300 represents the boundary of the listening environment. The primary speaker 202 and the slave speaker 204 are shown opposite each other within the listening environment and spaced apart by a fixed distance Dl along the horizontal x-axis. Also shown is the desired sweet spot of the user 312 and this sweet spot is referred to as the target position 310. The distance Ll is the distance from the first detection device 206 to the user 312 and the distance L2 is the distance from the second detection device 208 to the user 312 and the distance d is the distance along the vertical y-axis from the user to the horizontal x-axis between the primary speaker 202 and the slave speaker 204.

[0044] The sensors 120 in the detection devices 206, 208 detect objects within the listening environment, including the user 312 and one or more gestures made by the user 312. The electronic device will detect moving objects and identify which moving objects are people. Once the detected objects are identified as people, the electronic device will track all the human objects and wait to detect a first gesture 314 from one of the human objects. If the first gesture 314 is performed by a human object, the system will track that human object and wait for the tracked human object to perform a second gesture 316. When the second gesture is performed, the position of the person indicates to the electronic device the target position for the sweet spot setting.

[0045] Frequent switching of the sweet spot can have an adverse effect on the performance of the speakers, therefore, to avoid false switching of the sweet spot and prevent false beam adjustment, the user can perform two gestures. The first gesture is to wake up the electronic device. The second gesture is to lock the target position of the sweet spot.

[0046] The first gesture 314 is associated with waking up the electronic device to alert the device that the user 312 wants to adjust the sweet spot. For example, the first gesture 314 can be a wave with the hand. The first gesture 314 can be detected by one or both of the first detection device 206 of the primary speaker 202 and the second detection device 208 of the slave speaker 204. Upon detecting the first gesture 314, the sensors 120 on both the primary speaker 202 and the slave speaker 204 will wake up and track the user 312.

[0047] When the user wants to lock the sweet spot, a second gesture 316 different from the first gesture 314 is performed. When the user 312 performs the second gesture 316, both the primary speaker 202 and the slave speaker 204 will lock the position of the user 312. The second detection device 208 on the slave speaker 204 sends its lock information to the first detection device 206 on the primary speaker 202. Upon receiving the lock information from both detection devices 206, 208, the electronic device calculates and configures the sweet spot.

[0048] Figure 4 is a flowchart of a general method 400 of automatic beamforming. The method starts with the electronic device in a waiting state 402. The electronic device detects 404 at least one object in the listening environment. The electronic device detects 406 a first gesture and enters an active state. A timer with a predetermined time limit is set 408 and the active state lasts until the user performs a second gesture or the timer expires. During the active state, the electronic device tracks the user (the object from which the first gesture was detected).

[0049] During the predetermined time period, the user can stay in their existing position or, alternatively, move to a position in the listening environment where the sweet spot is to be set. Because the first gesture has been detected, the electronic device is in the active state and will track 410 the user until a certain point in time, or the timer expires 412, or the user performs a second gesture and it is detected 414

[0050] the second gesture.

[0051] When the timer expires and the second gesture has not been detected, the electronic device returns to the waiting state 402. When the user performs the second gesture within the predetermined time period and the one or more detection devices detect the second gesture 414, the user position is locked 416 into the electronic device as the target position for the sweet spot.

[0052] After the target position is locked 416, the electronic device has sensor information to calculate 418 the target position of the sweet spot for the purpose of automatic beamforming. Reference is made back to Figure 3, the target position 310 coincides with the position of the user 312 when the second gesture is detected, and is the desired sweet spot or target position 310. The distance D1 between the speakers is known and is a baseline parameter. The angle a1 is the angle between the first detection device 206 and the target position 310, and is derived from the sensor 120 when the second gesture is detected and the target position is locked. The angle a2 is the angle between the second detection device 208 and the target position 310, and is derived from the sensor 120 when the second gesture is detected and the target position is locked.

[0053] Using trigonometric functions, the distances L1 and L2 can be calculated as follows:

[0054] tan(a1) = d / d1; (1)

[0055] tan(a2) = d / d2; (2)

[0056] d1 + d2 = D1; (3)

[0057] d = D1 / (1 / tan(a1)) + D1 / (1 / tan(a2)) (4)

[0058] The distance d is the distance from the baseline to the target position 310. Knowing d, L1 and L2 can be calculated as follows:

[0059] L1 = d / sin(a1); and (5)

[0060] L2 = d / sin(a2). (6)

[0061] Referring again to Figure 4 , the electronic device configures the beam angle 420 for each of the primary and slave speakers by adjusting the sound beam based on the detected object angles a1, a2. The electronic device configures the beam intensity and propagation delay 422 for the primary and slave speakers by adjusting the sound beam intensity and propagation delay based on the calculated distances L1, L2. Beamforming is performed 424 using the configured beam angle, beam intensity, and propagation delay. After beamforming, the electronic device returns to its "wait" state 402.

[0062] When there are multiple people in the listening environment, the electronic device will recognize the first gesture made by the user to correctly identify the user as the object to be tracked. Upon detecting the first gesture, the electronic device has been alerted that the user wants to configure a sweet spot. The electronic device, upon detecting the second gesture, locks the current position of the user as the sweet spot.

[0063] There can be multiple modes of operation associated with the electronic device. For example, a scan lock mode, a scan follow mode, and a party mode. The modes of operation are configured on the electronic device using a mobile application, for example. The user selects the mode and once selected, the detection device will detect the object, track the movement and detect gestures associated with the mode settings.

[0064] The scan lock mode can provide a stable sound quality. Figure 5 An example 500 of the scan lock mode is shown. A user 512 performs a first gesture and a second gesture to set a target position 510 of the sweet spot and an operation occurs to configure the audio beam adjustment settings to the target position 510. Once set, the beam adjustment settings of the sweet spot target position 510 will remain at the target position even when the user 512 moves to a location such as location 514. The beamforming is directed to the sweet spot target position 510 and remains until the point in time when the user performs the first gesture and the second gesture again.

[0065] An example 600 for the scan follow mode is shown in Figure 6 In the scan follow mode, the target position of the sweet spot 610 follows the location of the tracked user. In this mode, a user 612 performs a first gesture and a second gesture. Once the system recognizes the two gestures, the electronic device will track the user 612, constantly adjusting the beamforming configuration of the sweet spot so that the sound beam 620 follows the user 612 to multiple locations 614, 616, 618. In the "party" mode, there is effectively no target position of the sweet spot. The loudspeaker will broadcast audio to the entire 360°.

[0066] To perform the methods described in Figure 4 The first and second loudspeakers need to be able to detect the angle of a human user to indicate the beam angle and distance in order to adjust the beam strength and propagation delay in order to perform the methods described in

[0067] As mentioned above, the sensor 120 can be camera based, such as an RGB camera or a thermal camera. RGB type cameras are very effective for recognizing and tracking people. However, there can be privacy issues since very clear images from the private network can be exposed. A thermal camera is an alternative that is able to detect and track people without showing clear images. However, the cost of a thermal camera is much higher than an RGB camera.

[0068] Position node tracking using DVS

[0069] Another alternative to the sensor 120 is a dynamic vision sensor (DVS). The DVS detects object motion with sufficient resolution while hiding the clear image of the object and solving the privacy problem. In addition, the cost is lower than a thermal camera. Each of the main speaker 202 and the slave speakers 204 can have four DVSs to create a 360° field of view (FOV) to detect objects and gestures.

[0070] Performing object and gesture detection with a DVS can be done by a fusion technique that combines several event frames with different timestamps into a single frame that provides an object time image. For object detection, the frame is fed to the processor of a neural network and object recognition is performed by a pre-trained model for object recognition. Gesture detection is performed by another pre-trained model for object classification. For gesture detection, the area of the gesture is smaller than the area of the object, so the image is zoomed in for more accurate detection.

[0071] However, the DVS is an event-based sensor that only senses light changes on each pixel and sends event packets with a timestamp and pixel location. The electronic device collects the event packets and recombines them into image frames for fusion. Since the DVS only detects moving objects, it can lose track if the object stops moving. Therefore, position node tracking is proposed as an alternative to the tracking algorithm applied to RGB-type cameras.

[0072] Reference Figure 7 A method 700 for position node tracking for the electronic device 100 is described, where the sensor 120 in the detection device for detecting objects and gestures in the listening environment is one or more DVSs. For a 360° field of view (FOV) of the speakers, the detection device will consist of four DVSs. The position node tracking is applied to the object and gesture detection steps discussed earlier in reference to Figure 4 The detection device scans the listening environment and, upon detecting 702 one or more moving objects, the electronic device creates 704 one or more temporary position nodes. The number of temporary position nodes created will depend on the bounding box associated with each object detected, and whether any bounding boxes overlap.

[0073] Whenever a temporary location node is created 704, the electronic device searches the existing pool of nodes 706 for location nodes created in a previous time stamp to determine if the temporary location node is in the vicinity of any existing location nodes currently present in the pool of nodes 708. If any temporary node is within a predetermined range of a node already present in the pool of nodes 710, the electronic device considers the temporary node to be an existing node that has moved to a new location, and creates a new node 712, removes the existing node from the pool of nodes 714, and adds the new node to the pool of nodes 716. If no existing node is found in the vicinity 718, the temporary node is considered a new node. A new node is created 720 and added to the pool of nodes 716.

[0074] When a new node is added to the pool of nodes, a timer is set 722 for a predetermined time. Within that predetermined time, for example two minutes, the node remains active in the pool of nodes. The electronic device continues to track 724 any activity occurring within the nodes in the pool of nodes using the timer. If the timer for a node in the pool of nodes expires 726, it means that there has been no movement in the location node, and it can be assumed that the location node is no longer needed to be tracked. For example, the user can have left the area or can have fallen asleep. To conserve computing resources, the location node with the expired timer is removed 728 from the pool of nodes and is no longer tracked. A live person should have at least slight movement within the predetermined time, which is enough to trigger the DVS and reset the timer associated with the location node. The timer is the key to continuously tracking the user.

[0075] Figure 8 to Figure 11 The application of location node tracking according to the rules for location node tracking is illustrated. Using a timer instead of an object to track the location node is the first rule for location node tracking using the DVS. In Figure 8 In the middle, a location node 802 is created around two objects 804 with a timer set for a predetermined time, such as two minutes. If either object does not move for two minutes 806, the location node 802 will be removed from the pool of nodes. If at least one of the objects moves within the existing location node before the timer expires 808, the timer will be reset for another two minutes of the predetermined time, and the location node 802 remains active in the pool of nodes and tracking continues.

[0076] Now consider the case where an object moves outside of an existing location node but is still in the vicinity. Converting the object movement to a node trajectory to reduce the complexity of object tracking is the second rule for location node tracking using the DVS. In Figure 9In this case, a location node 902 is created around a single object 904. If the object 904 moves to a new location near an existing location node, a new location node 906 is created to replace the existing location node 902. The new node 906 is added to the node pool and the existing location node 902 is removed from the node pool.

[0077] The third rule for location node tracking using DVS is that for objects detected but not in proximity to each other, the electronic device tracks multiple nodes simultaneously. Figure 10 is the case where a location node 1002 is created around two objects 1004a and 1004b. When one object 1004b moves out of the location node 1002, a new location node 1006 is created and the original location node 1002 is kept. A location node only covers a certain range. Therefore, if the two objects are not within the range of each other, both nodes 1002, 1006 exist in the node pool simultaneously and are tracked by their respective timers.

[0078] The fourth rule for location node tracking using DVS is that once more than one object is in proximity to each other, only one of the nodes needs to be tracked, thus reducing the complexity of tracking. Figure 11 is the case where a location node 1102 is created for object 1104a and a location node 1106 is created for object 1104b and both are active in the node pool. Objects 1104a and 1104b can be moving and the DVS will detect when they are in proximity to each other. In this case, the nodes 1102 and 1106 for the two objects 1104a and 1104b will merge into one location node, e.g., 1102. The timer will be reset for node 1102 and this node will remain active in the node pool. As Figure 11 indicated by the dashed line in, the remaining location node 1106 is deleted from the node pool when its timer expires.

[0079] Using rules one to four, the electronic device can track all active nodes and zoom in on certain areas for gesture detection. This improves the accuracy of gesture detection while also solving the problem of losing track of objects that do not move.

[0080] Depth detection and auto-calibration

[0081] After detecting the user and determining that the user wants to adjust the sweet spot, the electronic device must determine the angle of the user in order to direct the sound beam, and must determine the distance to the user in order to adjust the beam strength and the propagation delay factor. The relative angle can be determined when each detection device locates the x-y coordinates of the position node on the sensor 120. However, the depth of the object cannot be determined from the values of the x-y coordinates. In most cases, a binocular camera is used to calculate the depth based on a known baseline. In the present example, using the first detection device 206 and the second detection device 208, the baseline can be determined without the need for a binocular camera. Although the present example is directed to a pair of beamforming speakers, it should be noted that it can also be applied to other systems that use depth detection for object detection.

[0082] Figure 12 A pair of speakers, the primary speaker 202 and the secondary speaker 204, are shown, each having a detection device 1202a, 1202b, respectively. After installation of the system, each detection device 1202a and 1202b has two to four DVS to generate a 180° / 360° FOV, and can detect or measure its own angle values al, a2 by locating an object in its FOV, taking the known Dl value, the measured al and a2 values, and feeding them into a trigonometric function, the electronic device can calculate the depths LI and L2, as described above with reference to Figure 3 However, it can be necessary to acquire the baseline Dl without having to measure it manually. For example, when the position of the detection devices has changed from the original installation. The system described hereinafter automatically calibrates the relative position of the detection devices and the baseline Dl of the electronic device.

[0083] Reference is now made to Figure 13 A plurality of light sources, for example LEDs configured on an LED ring 1300, are incorporated with the secondary speaker 204. The LEDs on the LED ring 1300 are located on the ring and produce a known pattern of light points. During calibration, the LEDs flash one by one, triggering the detection setup on the primary speaker. The LEDs can be made to flash simultaneously once using an LED tracking circuit. The detection device on the primary speaker captures the flashing light points to create an image of the detected light point pattern. The electronic device compares the captured image to images in a database of calibration patterns and determines the relative position and the baseline Dl (the distance between the primary speaker and the secondary speaker) from the comparison. The LEDs can be infrared, so they are not visible to the human eye.

[0084] Figure 14A And Figure 14B An example of the LED pattern captured by the detection device on the primary speaker when the primary speaker and the secondary speaker 204 are directly opposite each other but the secondary speaker 204 is at a different distance from the primary speaker is shown. In this case, the baseline Dl is longer than the baseline D2. Figure 14AIn the case where the master speaker (not shown) and the slave speaker 204 are separated from each other by a short distance, an image 1400 is created on the master speaker by capturing an image at the detection device on the speaker as the LEDs on the LED ring 1300 flash. The image 1400 is compared to known patterns and the relative distance between the master speaker and the slave speaker is determined.

[0085] In the case where the master speaker (not shown) and the slave speaker 204 are separated from each other by a short distance, an image 1400 is created on the master speaker by capturing an image at the detection device on the speaker as the LEDs on the LED ring 1300 flash. The image 1400 is compared to known patterns and the relative distance between the master speaker and the slave speaker is determined. Figure 14B Figure 14A In the case where the master speaker (not shown) and the slave speaker 204 are separated from each other by a long distance, an image 1402 is created as described above. The image 1400 from the master speaker detection device shows a pattern of LEDs detected by the master speaker that is more sparse and spread over a larger area than the more dense pattern shown in the image 1402 in the case where the master speaker and the slave speaker are separated by a short distance. Figure 14B

[0086] Figure 15A and Figure 15B shows an example of the LED pattern captured by the master speaker (not shown) when the slave speaker 204 is not directly opposite the master speaker horizontally and at a different relative angle to the master speaker. In Figure 15A In the case where the slave speaker 204 is positioned at an angle above the master speaker on one side, the LED pattern 1500 will be detected on the upper region of the master speaker. In Figure 15B In the case where the slave speaker is positioned at an angle below the master speaker on one side, the LED pattern 1502 is detected on the bottom region of the master speaker.

[0087] The electronic device has a database of known calibration patterns stored in memory relating to specific distances between the master speaker and the slave speaker. For example, a calibration pattern for a pair of master and slave speakers separated by two feet is stored, a calibration pattern for a pair of master and slave speakers separated by five feet is stored, a calibration pattern for a pair of master and slave speakers separated by 10 feet is stored, and so on. When the detected pattern is compared to the calibration patterns, a match will indicate the distance and relative angle between the master speaker and the slave speaker. When no match is made to a calibration pattern in the database, the electronic device will apply interpolation to determine the distance and relative angle.

[0088] Figure 16 ​​is a flowchart 1600 of a method for depth detection and calibration. A slave speaker is activated from an LED ring on the speaker to cause the light sources (e.g., LEDs) to flash simultaneously and continuously once 1602. A detection device on the master speaker detects the light points 1604 and creates a detected pattern 1606. The detected pattern is compared to a database of known calibration patterns 1608. Based on the comparison results, a match, or applying an interpolation method, the distance and relative angle of the master speaker and slave speaker are determined 1610. The distance and relative angle determined in step 1610 are used to calculate and set 1612 the baseline parameter Dl. Trigonometric functions can be used to calculate the baseline parameter Dl as discussed earlier herein with reference to Figure 3 and Figure 12 as discussed.

[0089] In the foregoing specification, the disclosure has been described with reference to specific exemplary embodiments thereof. The specification and drawings are, accordingly, to be regarded as illustrative rather than restrictive, and modifications are intended to be included within the scope of the disclosure. The scope of the disclosure should, therefore, be determined not with reference to the description and drawings alone, but instead should be determined with reference to the appended claims and their legal equivalents, along with the full scope of equivalents to which such claims are entitled.

[0090] For example, the steps recited in any method or process claim can be executed in any order and are not limited to the specific order presented in the claims. Additionally, the components and / or elements recited in any device claim can be assembled or otherwise operatively configured in a variety of permutations, and are accordingly not limited to the specific configuration recited in the claims.

[0091] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments; however, any benefit, advantage, solution, or element that causes any particular benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature or element of any or all the claims.

[0092] The terms "comprises", "comprising", "includes", "including", "has", "having", "contains", "containing", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, composition, or apparatus that comprises, includes, has, contains a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article, composition, or apparatus. Other combinations and / or modifications of the above-described structures, arrangements, applications, proportions, elements, materials or components used in the practice of the present disclosure, other than those already described, can be used without departing from the spirit or scope of the disclosure.

Claims

1. A method for tracking location nodes of one or more mobile objects, comprising: tracking location nodes of one or more mobile objects, the tracking comprising scanning for one or more mobile objects, detecting one or more mobile objects, creating a temporary location node around one or more detected mobile objects, searching a pool of nodes for existing location nodes near the temporary location node, setting a timer for the existing location nodes near the temporary location node, the timer having a predetermined time limit, and setting a timer for temporary location nodes not near an existing location node, creating a new location node added to the pool of nodes and setting a timer associated with the new location node, the timer associated with the new location node having a predetermined time limit.

2. The method of claim 1, wherein an existing location node is removed from the pool of nodes when none of the objects in the existing location node move before the timer associated with the existing location node expires.

3. The method of claim 1, wherein the timer associated with an existing location node is reset when at least one of the objects in the existing location node moves before the timer associated with the existing location node expires, and the existing location node remains in the pool of nodes.

4. The method of claim 1, wherein a new location node is created to replace an existing location node when at least one of the objects in the existing location node moves outside the existing location node but remains near the existing location node, and a timer associated with the new location node is set, the existing location node is removed from the pool of nodes.

5. The method of claim 1, wherein a new location node is created to replace an existing location node when at least one of the objects in the existing location node moves to a location outside the existing location node, a timer associated with the new location node is set, and the existing location node remains for the duration of the timer associated with the existing location node.

6. The method of claim 1, wherein when there are two or more existing location nodes in the pool of nodes and one or more objects are near each other, the two or more existing location nodes are merged into one location node, a timer associated with the one location node will be reset, and the other existing location nodes will be removed from the pool of nodes.

7. A system for tracking location nodes of one or more mobile objects, comprising: a processor; a detection device, the detection device having at least one dynamic vision sensor for detecting mobile objects; the processor comprising computer readable instructions stored in a non-transitory memory, the computer readable instructions for: scanning for mobile objects using the detection device; detecting one or more mobile objects using the detection device; creating a temporary location node around one or more mobile objects; searching a pool of nodes for existing location nodes near the temporary location node; setting a timer associated with the existing location node when it is determined that the temporary location node is near the existing location node; creating a new location node when it is determined that the temporary location node is not near the existing location node; adding the new location node to the node pool; setting a timer associated with the new location node; tracking location nodes whose timers have not expired; and deleting location nodes whose timers have expired.

8. The system of claim 7, wherein the processor further comprises instructions to determine when none of the objects in an existing location node move before the timer associated with the existing location node expires and delete the existing location node from the node pool.

9. The system of claim 7, wherein the processor further comprises instructions to determine when at least one of the objects in an existing location node moves before the timer associated with the existing location node expires and reset the timer associated with the existing location node.

10. The system of claim 7, wherein the processor further comprises instructions to determine when at least one of the objects in an existing location node moves outside of the existing location node but remains near the existing location node, create a new location node to replace the existing location node, set a timer associated with the new location node, and delete the existing location node from the node pool.

11. The system of claim 7, wherein the processor further comprises instructions to determine when at least one of the objects in an existing location node moves to a location outside of the existing location node, create a new location node to replace the existing location node, set a timer associated with the new location node, and continue to track the duration of the timer associated with the existing location node for the existing location node.

12. The system of claim 7, wherein the processor further comprises instructions to determine when there are two or more existing location nodes in the node pool and one or more objects are near each other, merge the two or more existing location nodes into one of the existing location nodes, reset a timer associated with the one of the existing location nodes; and delete the other existing location nodes that were merged into the one of the existing location nodes from the node pool.

13. A method for tracking location nodes of one or more moving objects, comprising: scanning for moving objects; detecting one or more moving objects; creating a temporary location node around one or more moving objects; searching a node pool for an existing location node near the temporary location node; setting a timer associated with the existing location node when it is determined that the temporary location node is near the existing location node; creating a new location node when it is determined that the temporary location node is not near the existing location node; adding the new location node to the node pool; ​ setting a timer associated with the new location node; tracking location nodes for which the timer has not expired; and deleting location nodes for which the timer has expired.

14. The method of claim 13, wherein when none of the objects in an existing location node move before expiration of the timer associated with the existing location node, the method further comprises deleting the existing location node from the pool of nodes.

15. The method of claim 13, wherein when at least one of the objects in an existing location node moves before expiration of the timer associated with the existing location node, the method further comprises resetting the timer associated with the existing location node.

16. The method of claim 13, wherein when at least one of the objects in an existing location node moves outside of the existing location node but remains in the vicinity of the existing location node, the method further comprises: creating a new location node to replace the existing location node; setting a timer associated with the new location node; and deleting the existing location node from the pool of nodes.

17. The method of claim 13, wherein when at least one of the objects in an existing location node moves to a location outside of the existing location node, the method further comprises: creating a new location node to replace the existing location node; setting a timer associated with the new location node; and continuing to track the existing location node for the duration of the timer associated with the existing location node.

18. The method of claim 13, wherein when there are two or more existing location nodes in the pool of nodes and one or more objects are in close proximity to each other, the method further comprises: merging the two or more existing location nodes into one of the existing location nodes; resetting a timer associated with the one of the existing location nodes; and deleting the other existing location nodes merged into the one existing location node from the pool of nodes.

Images