Earphone camera-based interaction method, earphone, and storage medium
By acquiring image and inertial sensing data from the headphones to determine area priorities, the problem of broadcast errors caused by random decoding of QR codes in the headphone scanning solution is solved, improving the response efficiency of QR code recognition and the navigation experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GOERTEK INC
- Filing Date
- 2026-06-03
- Publication Date
- 2026-07-14
Smart Images
Figure CN122387409A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of headphone technology, and more particularly to an interaction method based on a headphone camera, a headphone, and a storage medium. Background Technology
[0002] In related technologies, headphones are used to identify target objects and provide voice prompts. The implementation involves global feature point extraction and matching of images captured by dual headphones, calculating and extracting the overlapping area of the two images' fields of view, performing target object recognition and detection within this overlapping area, and finally triggering corresponding audio interactive feedback based on the target object recognition result.
[0003] However, in museum and scenic area guided tour scenarios, a single display case may have multiple QR codes for exhibits, a wall may have multiple QR codes for attraction information, and display boards may simultaneously display guide codes, payment codes, and reservation codes. Current headphone scanning solutions randomly decode the first detected QR code, which can easily lead to abnormal playback situations where "the user points the headphone at exhibit A0, but the system reads information about exhibit B," negatively impacting the guided tour experience.
[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention
[0005] The main purpose of this application is to provide an interaction method based on an earphone and camera, an earphone, and a storage medium, which aims to solve the technical problem of random decoding when multiple QR codes appear simultaneously, resulting in voice broadcast errors.
[0006] To achieve the above objectives, this application proposes an interaction method based on an earphone camera, wherein the earphone includes at least one earphone unit equipped with a camera module and an inertial measurement unit, and the method includes: The image output by the camera module of the headphone unit is acquired, and the inertial sensing data output by the inertial measurement unit is acquired, wherein the timestamp of the inertial sensing data is the same as that of the image. The region priority of each preset region in the image is determined based on the inertial sensing data; According to the region priority, the QR code detection process is executed in each of the preset regions, so that the QR code detection process is terminated when the target QR code is detected. Output the voice information corresponding to the target QR code.
[0007] In one embodiment, the inertial sensing data includes triaxial acceleration and triaxial angular velocity, and the step of determining the region priority of each preset region in the image based on the inertial sensing data includes: Determine the row vector composed of the triaxial acceleration and the triaxial angular velocity, and calculate the transpose vector corresponding to the row vector; Obtain the weight vector and bias of each preset region; Calculate the probability value of each preset region based on the weight vector, the bias, and the transpose vector; The priority of the region is determined based on the probability value and the preset region to which the probability value is bound.
[0008] In one embodiment, the step of calculating the probability value of each of the preset regions based on the weight vector, the bias, and the transpose vector includes: Calculate the product of the weight vector and the transpose vector; The sum of the product and the bias is determined as the linear score for each of the preset regions; The probability values of each preset region are determined after the linear score is subjected to exponential operation and normalization.
[0009] In one embodiment, before the steps of acquiring the image output by the camera module of the earphone unit and acquiring the inertial sensing data output by the inertial measurement unit, the interaction method based on the earphone camera further includes: Acquire the QR code image and corresponding inertial data collected by the earphone unit during the registration stage; Based on the region where the QR code is located in the QR code image, the inertial data, and the preset algorithm, the weight vector and the bias corresponding to each preset region in the headphone unit are calculated.
[0010] In one embodiment, the earphone unit includes a first earphone unit and a second earphone unit, and the target QR code includes a first QR code collected by the first earphone unit and a second QR code collected by the second earphone unit. The step of performing a QR code detection process in each of the preset areas according to the area priority, so that if the target QR code is detected, the QR code detection process is terminated and the voice information corresponding to the target QR code is output, includes: If there is a difference between the first QR code and the second QR code, the priority of the first QR code and the second QR code is determined based on the attribute information of the first QR code and the second QR code; The QR code with the highest priority is identified as the target QR code; Output the voice information corresponding to the target QR code.
[0011] In one embodiment, after the step of performing a QR code detection process in each of the preset areas according to the area priority, so as to terminate the QR code detection process when a target QR code is detected, the interaction method based on the earphone camera further includes: If the first QR code and the second QR code are the same QR code, then output the voice information corresponding to the target QR code.
[0012] In one embodiment, the step of performing a QR code detection process within each of the preset regions according to the region priority, so as to terminate the QR code detection process when a target QR code is detected, includes: Determine the detection parameters corresponding to the region priority, wherein the detection parameters are different for different region priorities; Based on the detection parameters and the region priority, a QR code detection process is executed for each of the preset regions, so that the QR code detection process is terminated when a target QR code is detected.
[0013] In one embodiment, the target QR code corresponds to multiple voice messages of different durations, and the step of outputting the voice message corresponding to the target QR code includes: The average moving speed of the headphone unit is determined based on the inertial sensing data over a preset time period; The duration of the target QR code in the image is determined based on the average moving speed; Based on the duration, the target voice information corresponding to the target QR code is determined and the target voice information is output.
[0014] In addition, to achieve the above objectives, this application also proposes an earphone, the earphone comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the earphone-camera-based interaction method as described above.
[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the interaction method based on the headphone camera described above.
[0016] One or more technical solutions proposed in this application have at least the following technical effects: The system acquires images output from the camera module of the headphone unit and inertial sensing data output from the inertial measurement unit at the same time as the image acquisition. Based on the inertial sensing data, it determines the region priority of each preset region in the image, and then performs a QR code detection process in each preset region according to the region priority. The QR code detection process terminates when a target QR code is detected. Finally, it outputs the voice information corresponding to the target QR code. By using inertial sensing data that is strictly synchronized with the image acquisition time, it reflects the user's head orientation and device pointing when wearing the headphone unit, and then reasonably infers the user's visual attention area. It establishes a priority order for different preset regions in the image that conforms to the user's actual usage intention, solving the problem of the system randomly selecting the decoding object when multiple QR codes appear in the video screen at the same time, which causes voice playback errors. At the same time, it immediately terminates the subsequent detection operation after successfully detecting the target QR code, reducing redundant computing overhead and improving the response efficiency of QR code recognition. Attached Figure Description
[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a flowchart illustrating the first embodiment of the interaction method based on an earphone camera in this application; Figure 2 This is a schematic diagram of the preset area in the interaction method based on the headphone camera in this application; Figure 3 This is a schematic diagram of the area of the first QR code in the image in the interaction method based on the headphone camera of this application; Figure 4 This is a schematic diagram of the area of the second QR code in the image in the interaction method based on the headphone camera of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the interaction method based on the headphone camera in the embodiments of this application.
[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.
[0022] In related technologies, headphones are used to identify target objects and provide voice prompts. The implementation involves global feature point extraction and matching of images captured by dual headphones, calculating and extracting the overlapping area of the two images' fields of view, performing target object recognition and detection within this overlapping area, and finally triggering corresponding audio interactive feedback based on the target object recognition result.
[0023] However, in museum and scenic area guided tour scenarios, a single display case may have multiple QR codes for exhibits, a wall may have multiple QR codes for attraction information, and display boards may simultaneously display guide codes, payment codes, and reservation codes. Current headphone scanning solutions randomly decode the first detected QR code, which can easily lead to abnormal playback situations where "the user points the headphone at exhibit A0, but the system reads information about exhibit B," negatively impacting the guided tour experience.
[0024] Based on this, this application provides a solution that utilizes inertial sensing data that is strictly synchronized with the image acquisition time to reflect the user's head orientation and device pointing when wearing the headphone unit, thereby reasonably inferring the user's visual attention area and establishing a priority order for different preset areas within the image that conforms to the user's actual usage intention. This solves the problem of the system randomly selecting the decoding object when multiple QR codes appear in the video screen at the same time, causing voice playback errors. At the same time, it immediately terminates subsequent detection operations after successfully detecting the target QR code, reducing redundant computational overhead and improving the response efficiency of QR code recognition.
[0025] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or headphones capable of performing the above functions. The headphones can be TWS (True Wireless Stereo) headphones, over-ear headphones, or open-back headphones. The following description uses TWS headphones as an example to illustrate this embodiment and the subsequent embodiments.
[0026] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.
[0027] This application provides an interaction method based on an earphone camera, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the interaction method based on an earphone camera according to this application.
[0028] In this embodiment, the interaction method based on the headphone camera includes steps S10~S40: Step S10: Acquire the image output by the camera module of the headphone unit and the inertial sensing data output by the inertial measurement unit.
[0029] The inertial sensing data has the same timestamp as the image.
[0030] The aforementioned headphones include at least one headphone unit equipped with a camera module and an inertial measurement unit (IMU). The camera module is a miniature image acquisition module integrated into the headphone. The inertial sensing data is physical quantity data acquired by the inertial measurement unit (IMU) built into the headphone unit, including three-axis acceleration and three-axis angular velocity, used to calculate the device's spatial attitude and motion state. The acquired inertial sensing data and the image data share the same timestamp, meaning they were acquired at the same timestamp.
[0031] In this embodiment, an image can be acquired after responding to a QR code trigger command. The QR code trigger command is a control command used to initiate the QR code scanning interaction process on the earphone. Triggering methods include hardware triggering and software triggering, such as generation through physical buttons or touch-sensitive areas on the earphone itself, or receiving a QR code trigger command sent by a smart terminal paired with the earphone. After responding to the QR code trigger command, the main control processor built into the earphone unit simultaneously sends a unified clock synchronization signal to the camera module and the inertial measurement unit to eliminate clock drift errors between different sensors. Then, after acquiring an image, the camera module adds a hardware timestamp corresponding to the current clock to the image attribute information. Similarly, the inertial measurement unit adds a hardware timestamp from the same clock source to the header of the data packet each time it samples inertial data, ensuring that both correspond to the device state at the same moment. Therefore, after acquiring an image, inertial sensing data from the same time is selected based on the timestamp of that image.
[0032] For example, when a user wearing headphones looks at multiple QR codes on exhibits in a display case, the camera module captures images, while the inertial measurement unit samples acceleration and angular velocity data at a frequency of 100Hz. The main control processor pairs the images and inertial data with timestamp differences within 5ms to obtain inertial sensing data at the same time as the image acquisition.
[0033] Optionally, to reduce the amount of data processing, i.e., reduce the CPU utilization and power consumption of the headphone unit and extend the battery life, a frame of image can also be acquired. That is, in response to the scanning trigger command, a frame of image output by the camera module of the headphone unit is acquired, along with inertial sensing data at the same acquisition time as the image.
[0034] Understandably, headphones typically lack a screen display, making it impossible to directly adjust the scanning position during shooting or scanning. Direct image capture can easily result in issues such as incomplete QR code inclusion, out-of-focus images, and repeated captures. This embodiment addresses this by acquiring both the image and inertial sensor data captured at the same time. By obtaining this inertial sensor data, the system accurately reflects the user's head posture and device orientation at the moment the video frame is captured, thus accurately identifying the user's true area of focus and providing accurate data for determining area priority in subsequent steps.
[0035] It is understandable that when shooting through one of the earphone units, only the image output by that earphone unit is captured. However, when shooting through both ears, after responding to the QR code trigger command, the first image captured by the first earphone and the second image captured by the second earphone at the same time as the first image are directly acquired.
[0036] Furthermore, before step S10, the location information of the headphones can be obtained, and the voice information of the items at the current location can be obtained from the cloud based on the location information. For example, if the current location is exhibition hall A, the text information of all exhibits in exhibition hall A can be crawled from the Internet, and the corresponding voice information can be generated based on the text information and stored locally.
[0037] Step S20: Determine the region priority of each preset region in the image based on inertial sensing data.
[0038] In this embodiment, the preset region refers to multiple non-overlapping regions, such as a nine-square grid, concentric circle regions, or horizontally layered regions, pre-divided in the image's coordinate system. These regions are used to divide the image into different detection units. Region priority refers to the detection order weight assigned to each preset region based on the likelihood of user attention; regions with higher priority are detected for QR codes first.
[0039] As an optional method for determining area priorities, attitude calculation can be performed on inertial sensor data to obtain the spatial orientation of the headphone unit. This orientation is then mapped to the image coordinate system to determine the degree of overlap between different areas and the user's line of sight, thereby assigning priorities. Specifically, attitude calculation is performed on synchronous inertial sensor data. A complementary filtering algorithm is used to fuse static attitude data from the accelerometer and dynamic attitude data from the gyroscope to calculate three Euler angles for the headphone unit: pitch, yaw, and roll. Pitch represents the angle of vertical head movement, and yaw represents the angle of horizontal head movement. Next, a mapping relationship is established between the image coordinate system and the spatial orientation of the headphone unit. The center of the image is defined as directly in front of the head, the top and bottom edges of the image correspond to preset pitch angle ranges, and the left and right edges correspond to preset yaw angle ranges. Based on the calculated Euler angles, the corresponding position of the user's head pointing in the video image is determined. The Euclidean distance between the center of each preset area and this pointing position is calculated; areas closer to this position have higher priority, and areas farther away have lower priority.
[0040] For example, please refer to Figure 2 Taking the left earphone unit as an example, Figure 2 The image is divided into five preset regions: inner middle region, inner upper region, inner lower region, outer region, and occluded region. When the user's head is pointing directly at the center of the image, the Euclidean distance between the center region and the pointing position is 0, meaning the inner middle region has the highest priority. The inner upper and inner lower regions are next in priority, while the outer region is the furthest away and has the lowest priority. The occluded region is usually obscured by the user's head or cheek, so this region is usually not compared in terms of priority, or it is assumed that this region always has the lowest priority.
[0041] As an alternative implementation for determining region priority, attitude calculation can be performed on inertial sensing data to obtain the spatial orientation of the headphone unit. Simultaneously, considering the central field of vision characteristics of the human visual system, the user's central and peripheral visual field ranges can be determined. The screen is then divided into multiple preset regions corresponding to the central and peripheral visual fields: the central visual field region, the upper peripheral region, the lower peripheral region, the left peripheral region, and the right peripheral region. Finally, basic weights are assigned to the preset regions corresponding to different visual fields, with the central visual field region having a higher basic weight than the peripheral visual field region. The weights of each region are then fine-tuned based on slight shifts in head orientation, with regions having higher weights having higher priority. For example, when the user's head orientation shifts slightly to the right, the weight of the central visual field region remains the highest, while the weight of the right peripheral region is slightly increased.
[0042] In addition, the probability priority algorithm of the regression model can be used to learn the statistical regularity between IMU data and user attention areas from massive amounts of real user data, so that the model can automatically adapt to the head posture habits and headphone wearing differences of different users.
[0043] It is understandable that the two headphone units are usually photographed symmetrically on the left and right sides, so the regions of the two images have the same symmetrical priority.
[0044] Step S30: Perform the QR code detection process in each of the preset areas according to the area priority, so that the QR code detection process is terminated when the target QR code is detected.
[0045] Here, a QR code refers to a matrix-style QR code that can be recognized by standard QR code decoding algorithms and contains or is associated with broadcastable information such as text, links, and numerical values. The QR code detection process refers to the computer vision algorithm process of identifying QR codes and decoding their content from an image region through steps such as image preprocessing, feature extraction, and pattern matching. A target QR code is one that can be successfully decoded by the system and contains valid voice information associated with it. Terminating the QR code detection process means immediately stopping all ongoing QR code detection operations, clearing all pending detection tasks, and releasing occupied computing resources.
[0046] In this embodiment, the main control processor generates a detection task queue based on region priority, with the order of tasks in the queue corresponding to their priority from highest to lowest. Then, the first detection task is retrieved from the head of the queue, and image data for the corresponding preset region is extracted. A QR code detection process, including grayscale conversion, binarization, edge detection, contour extraction, QR code localization, and decoding, is then performed on this image data. If no valid QR code is detected in that region, the next priority detection task is retrieved from the queue, and the above detection process is repeated until all regions are detected or the target QR code is detected.
[0047] Optionally, a parallel hierarchical detection mechanism can be used to simultaneously execute detection tasks within the same priority group. Specifically, all preset regions are divided into three priority levels: a first priority group, a second priority group, and a third priority group, corresponding to high, medium, and low priorities, respectively. Then, QR code detection tasks for all regions within the high-priority group are started simultaneously, with each task assigned an independent computation thread. If no valid QR code is detected in any region within the high-priority group, the detection tasks for the medium-priority group are started. If no valid QR code is detected in the medium-priority group, the detection tasks for the low-priority group are started. If a target QR code is detected in any region within any group, all running and pending detection tasks are immediately terminated.
[0048] Optionally, when executing the QR code detection process, different detection parameters can be set based on the priority of different regions. These detection parameters include resolution, frame rate, etc. Therefore, the detection parameters corresponding to the region priority can be determined first, and then the QR code detection process can be executed for each preset region based on the detection parameters and region priority. Different region priorities result in different detection parameters. For example, a user wearing dual-ear TWS earphones enters the exhibition hall and looks at the QR codes of three exhibits displayed on the showcase. They are located in the inner middle region, the inner upper region, and the outer region, respectively. The calculated region priorities and corresponding probabilities are: inner middle (0.82, high priority) > inner upper (0.15, low priority) > outer region (0.02, low priority). At this time, the main control processor assigns detection parameters to each region according to the parameter mapping table: inner middle: 1280×720 resolution, 30fps frame rate, standard detection threshold 0.6; other regions are 320×180 resolution, 5fps frame rate, high detection threshold 0.8.
[0049] Optionally, if the left and right earphone units capture images simultaneously, and the preset regions of the two images are symmetrical and identical, then the preset region of one of the images can be selected for QR code detection. If the preset regions are not symmetrical, then one image can be randomly selected for detection, or priority detection can be performed on the preset regions in the two images, and the region with the higher priority among the two images can be selected for detection.
[0050] Furthermore, during the QR code detection process, the main control processor monitors the return results of each detection task in real time. When any detection task returns a successfully decoded target QR code, it immediately sends an interrupt signal to all running detection threads, forcibly terminating their execution, clearing all pending tasks in the detection task queue, and releasing image data buffers and computing resources. Additionally, when any detection task returns a successfully decoded target QR code, it can immediately stop submitting new detection tasks to the detection thread pool and clear all pending tasks in the task queue.
[0051] This embodiment prioritizes QR code detection, ensuring that areas most likely to attract user attention are detected first. This avoids the system randomly selecting detection areas, which could lead to unsuitable voice prompts. The detection process terminates immediately upon detecting a target QR code, preventing further detection of QR codes in other areas and avoiding voice prompt conflicts and errors caused by multiple QR codes being decoded simultaneously.
[0052] Step S40: Output the voice information corresponding to the target QR code.
[0053] In this embodiment, the voice information output process involves playing the text information or associated audio file obtained from decoding the QR code to the user through the headphone unit's speaker. The corresponding voice information can be retrieved from local storage or a cloud server based on the identifier information obtained from decoding the target QR code, and then played through the headphone unit's speaker. Alternatively, a text-to-speech (TTS) engine can be invoked to generate the corresponding voice information based on the identifier information obtained from decoding the target QR code, and then played through the headphone unit's speaker. For example, when the main control processor successfully decodes the attraction's QR code in the detection center area, it immediately stops submitting new detection tasks, clears other area detection tasks in the task queue, waits for the currently running upper half-area detection task to complete, and then invokes the cloud-based TTS engine to generate an audio explanation of the attraction, which is then played to the user through the headphone speaker.
[0054] Specifically, when outputting voice information, the system can parse the target QR code information, search for relevant audio stored locally in the headphones, and match the filename corresponding to the QR code information locally. The matched audio file is then decoded and played using the headphones' audio codec. The headphones can have relevant audio files pre-installed at the factory or download audio files in real-time when connected to the internet. For example, in tourist attractions, museums, and exhibition halls, exhibits typically have QR code labels. After recognizing the exhibit's QR code, the headphones can play an introductory audio message based on that code.
[0055] As another alternative implementation, the QR code can be uploaded to a smart terminal such as a mobile phone. After the smart terminal matches the corresponding voice information, the audio can be played simultaneously in the first and second earphones.
[0056] This embodiment provides an interaction method based on an earphone camera. It constructs a QR code priority detection mechanism based on user head posture data to solve the problem of voice broadcast errors caused by random decoding of multiple QR codes when they appear on the screen at the same time. It also reduces redundant image detection calculation overhead, lowers the operating power consumption of the earphone unit, and improves the response speed and interaction smoothness of QR code recognition.
[0057] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment can be referred to the above description, and will not be repeated hereafter. Based on this, the inertial sensing data includes triaxial acceleration and triaxial angular velocity, and step S20 includes steps S21~S24: Step S21: Determine the row vector composed of the three-axis acceleration and the three-axis angular velocity, and calculate the transpose vector corresponding to the row vector.
[0058] In this embodiment, a row vector refers to a 1×6 one-dimensional array of IMU row vectors formed by horizontally arranging six physical quantities—three-axis acceleration and three-axis angular velocity—that are strictly synchronized with the image frame acquisition time, in a preset fixed order. This vector is the standardized input format of the softmax regression model. Vector transpose refers to interchanging the rows and columns of the row vector, converting it into a 6×1 column vector through linear algebra operations, to satisfy the subsequent matrix dot product operation requirements.
[0059] Specifically, the extracted inertial data includes triaxial acceleration values ( ) and triaxial angular velocity values ( Then, following the exact same order as in the model pre-training phase, the six values are sequentially filled into a 1×6 array to form an IMU row vector. Subsequently, the IMU row vector is transposed to convert it into a 6×1 column vector, so that the dimension of the input vector matches the dimension of the weight vector, ensuring that the subsequent dot product operation can be legally executed.
[0060] For example, the acquired synchronous IMU data is =0.21m / s² =9.58m / s² =0.12m / s² =0.05rad / s =-0.02rad / s When the value is 0.01 rad / s, the generated IMU row vector is [0.21, 9.58, 0.12, 0.05, -0.02, 0.01], and the transposed column vector is [[0.21], [9.58], [0.12], [0.05], [-0.02], [0.01]].
[0061] Step S22: Obtain the weight vector and bias of each preset region.
[0062] The weight vector is a 1×6 row vector corresponding to each preset region. Each element in the vector represents the contribution weight of the corresponding IMU component to the user's attention probability in that region. This vector is the core parameter of the model obtained through pre-training with large-scale user data. The bias vector is a 1×N row vector composed of the bias values of all preset regions in order (N is the total number of preset regions). Each bias value is used to adjust the original score baseline of the corresponding region to compensate for the natural differences in human visual system attention to different regions.
[0063] Understandably, before step S10, it is necessary to acquire the QR code image and corresponding inertial data collected by the earphone unit during the registration stage. Then, based on the region where the QR code is located in the QR code image, the inertial data, and a preset algorithm, the weight vector and bias corresponding to each preset region in the earphone unit are calculated. The preset algorithm is as follows: , , , For the weight vector, Let x be the bias vector and x be the transpose vector. This is a probability value. The score is linear.
[0064] Therefore, before the headphone unit leaves the factory, hundreds of thousands of sets of synchronous IMU data, labeled with the user's actual focus areas, are collected in typical scenarios such as shopping, travel, and commuting, covering user groups of different ages, genders, and head sizes. Supervised learning algorithms are used to train a weight vector and corresponding bias values equal to the number of preset regions, and these parameters are then stored in the headphone's non-volatile Flash memory in binary format. During the headphone system's power-on initialization phase, the main control processor reads all weight vectors and bias values from the Flash memory at once and loads them into the on-chip cache, avoiding repeated access to slow external storage for each calculation. Finally, all bias values are concatenated into a complete bias vector according to the preset region numbering order, facilitating subsequent batch calculation of the raw scores for all regions.
[0065] For example, when the total number of preset regions is 5, 5 sets of 1×6 weight vectors ω1~ω5 and 5 bias values b1~b5 are obtained during pre-shipment training. During system initialization, these parameters are loaded into the cache and concatenated to form bias vectors [b1, b2, b3, b4, b5] for subsequent real-time calculation.
[0066] Step S23: Calculate the probability value of each preset region based on the weight vector, bias, and transpose vector.
[0067] The probability value refers to the 1×N row vector obtained after performing softmax normalization on the linear scores of each preset region. Each element is a probability value between 0 and 1, and the sum of all elements is always 1.
[0068] In this embodiment, calculating the probability value requires first calculating the product of the weight vector and the transpose vector, then determining the sum of the product and the bias as the linear score for each preset region, and finally determining the probability value for each preset region after exponential operation and normalization of the linear score. Specifically, the weight vector and transpose vector of each preset region are sequentially multiplied to obtain the linear transformation result of each region. The bias value at the corresponding position in the bias vector is then added to obtain the linear score of that region. The linear scores of all regions are combined sequentially into a linear score vector. The natural exponential operation is performed on each element in the linear score vector to obtain the exponential score of each region. At the same time, the sum of all exponential scores is calculated and used as the denominator for softmax normalization. The exponential score of each region is divided by this sum to obtain the attention probability of that region.
[0069] For example, the transpose vector is [[0.21], [9.58], [0.12], [0.05], [-0.02], [0.01]], the weight vector ω1=[0.18, 0.82, 0.09, 0.27, 0.21, 0.11], and the bias b1=0.42. The calculated dot product is 0.18×0.21+0.82×9.58+0.09×0.12+0.27×0.05+0.21×(-0.02)+0.11×0.01=7.89. The linear score after adding the bias is Z1=8.31. Similarly, Z2=3.07, Z3=2.61, Z4=-1.19, and Z5=-0.95 are calculated.
[0070] based on When calculating the probability values, first calculate the exponential scores: e^8.31≈4072, e^3.07≈21.5, e^2.61≈13.6, e^-1.19≈0.304, e^-0.95≈0.387, with a total of ≈4107.791. The final probability values are [0.9913, 0.0052, 0.0033, 0.000074, 0.000094].
[0071] Step S24: Determine the region priority based on the probability value and the preset region bound to the probability value.
[0072] In this embodiment, each probability value is bound to its corresponding preset region number. For example, the corresponding region numbers are 1 (inner middle), 2 (inner upper), 3 (inner lower), 4 (outer region), and 5 (occluded region). The generated key-value pair list is [(1,0.9913), (2,0.0052), (3,0.0033), (4,0.000074), (5,0.000094)]. The region priority sequence obtained after sorting by probability value is [1, 2, 3, 5, 4].
[0073] This embodiment provides an interaction method based on an earphone camera. It realizes the calculation of region priority based on a pre-trained model through a standardized linear algebra operation process, thereby automatically adapting to the different head posture habits and earphone wearing differences of different users, improving the accuracy and robustness of region priority allocation, and thus improving the recognition effectiveness of QR code recognition based on region priority.
[0074] Based on the first embodiment of this application, in the third embodiment of this application, the same or similar content as the first embodiment can be referred to the above description, and will not be repeated hereafter. Based on this, the earphone unit includes a first earphone unit and a second earphone unit, and the target QR code includes a first QR code collected by the first earphone unit and a second QR code collected by the second earphone unit. Therefore, step S40 includes steps S41~S43: Step S41: If there is a difference between the first QR code and the second QR code, determine the priority of the first QR code and the second QR code based on the attribute information and collection time sequence of the first QR code and the second QR code. Step S42: Determine the QR code with the highest priority as the target QR code; Step S43: Output the voice information corresponding to the target QR code.
[0075] In this embodiment, priority is a quantified sorting rule used to determine the playback priority of QR code objects. The higher the priority value, the more the QR code matches the user's current scanning needs, and the earlier it is played. Therefore, when there is a difference in the QR codes detected by the two earphone units, it is necessary to further determine which QR code meets the actual voice playback requirements. That is, first determine the priority of the first QR code and the second QR code, then select the one with the highest priority among the two QR codes as the target QR code, and at the same time output the voice information corresponding to the target QR code.
[0076] In this embodiment, the priority of the first and second QR codes can be determined based on their attribute information. The attribute information of the QR codes includes at least one of the following: the region where the QR code is located and the timing of its acquisition.
[0077] Therefore, it is necessary to obtain attribute information from the first and second QR codes, including the region where the QR code is located and at least one parameter in the QR code acquisition timing. For example, please refer to... Figure 2 , Figure 2 In the area shown, the detection priority is highest in the inner area, so the priority of the QR code detected in the inner area is higher than that of the QR code on the inner side.
[0078] As an optional implementation, when the attribute information is the region where the QR code is located, the first priority of the first QR code can be determined firstly based on the association between priority and region, and the region where the first QR code is located; similarly, the second priority of the second QR code can be determined based on the association between priority and region, and the region where the second QR code is located. Figure 2 The area shown represents images captured by the first earphone unit (left earphone) and the second earphone unit (right earphone). Figure 3 and Figure 4 As shown, the first QR code is located in the inner middle area, and its priority is 5. The second QR code is located in the inner lower area, and its priority is 4 or 3. In this case, the first QR code is determined as the target QR code.
[0079] As another optional implementation, when the attribute information is the acquisition time sequence of the QR code, the earlier the acquisition time sequence, the higher the corresponding priority. Therefore, when comparing priorities, the acquisition time sequence of the first QR code and the second QR code can be indirectly compared. If the acquisition time sequence of the first QR code is earlier than that of the second QR code, then the first QR code is determined as the target QR code, and vice versa.
[0080] This embodiment provides an interaction method based on an earphone camera. When there is a difference between a first QR code and a second QR code, the priority is determined directly or indirectly by the region where the QR code is located, or indirectly by the timing of the QR code acquisition, thereby obtaining the target QR code with the highest priority. Determining the priority by the region where the QR code is located allows QR codes in areas that are centered, unobstructed, and have low distortion to be identified as high-priority QR codes. This area also aligns with the user's actively adjusted wearing posture and deliberate alignment, accurately matching the scanning intent. Furthermore, in situations where the earphone lacks a screen preview, the user's true target can be accurately located in multi-QR code scenarios without manual selection or blind posture adjustment, improving both earphone battery life and the scanning interaction effect.
[0081] Based on the third embodiment of this application, in the fourth embodiment of this application, the same or similar content as the third embodiment can be referred to the above description, and will not be repeated hereafter. Furthermore, if the first QR code and the second QR code are the same, the voice information corresponding to the target QR code is output.
[0082] It is understandable that QR codes with the same content will generate the exact same hash value. Therefore, when two QR codes are identical, the first or second QR code can be used as the target QR code and the voice information can be played based on that target QR code.
[0083] In this embodiment, QR code similarity detection can be performed using a fast hash value comparison method. Specifically, after each earphone unit successfully decodes the target QR code, it immediately performs a SHA-256 hash operation on the decoded complete text content to generate a 256-bit hash value, and simultaneously records the decoding completion timestamp. Then, the generated hash value, its own device identifier, and the decoding timestamp are sent to the main system control unit. Upon receiving the first hash value, the main control unit immediately starts a preset timeout timer. Within the timeout period, if it receives a hash value from another earphone unit, and the two hash values are identical, the first and second QR codes are determined to be the same QR code, triggering the voice output process. If no hash value is received from another earphone unit within the timeout period, or the two hash values are different, the detection is deemed invalid, all cached hash values and timestamps are cleared, and the system waits for the next detection trigger.
[0084] Optionally, after successfully decoding the target QR code, each earphone unit sends the complete decoded text content, QR code version number, error correction level, and its own device identifier to the main system control unit. Upon receiving the first complete content, the main control unit starts a preset timeout timer. Then, within the timeout period, if the main control unit receives complete content from another earphone unit, it performs a precise byte-by-byte comparison of the two contents, simultaneously verifying whether the version numbers and error correction levels of the two QR codes are consistent. If the two contents match byte-by-byte and the version numbers and error correction levels are identical, they are determined to be the same QR code, triggering the voice output process; otherwise, the detection is deemed invalid, and all cached data is cleared.
[0085] For example, the first earphone unit decodes the QR code content of the scenic spot as "SCENIC-001 - Hall of Supreme Harmony in the Forbidden City - Chinese Explanation", with a QR code version number of 7 and an error correction level of M, and sends this information to the charging case's main control unit. 80ms later, the second earphone unit decodes the exact same content, version number, and error correction level. After comparing them byte by byte and confirming their consistency, it calls the cloud-based TTS engine to generate the corresponding audio explanation and distributes the audio data to both earphone units for synchronized playback.
[0086] This embodiment provides an interaction method based on an earphone camera. By using a distributed collaborative detection and cross-validation mechanism of dual earphone units, it not only solves the problem of random decoding errors when multiple QR codes appear simultaneously, but also further addresses the issue of false detections that are prone to occur when detecting QR codes in a single earphone, thereby improving the accuracy of QR code recognition and the robustness of the system in complex environments.
[0087] Based on the first embodiment of this application, in the fifth embodiment of this application, the content that is the same as or similar to that in the first embodiment can be referred to the above description, and will not be repeated hereafter. In addition, step S40 further includes steps S44-S45: Step S44: Determine the average moving speed of the headphone unit based on inertial sensing data over a preset time period.
[0088] The average moving speed of the earphone unit mentioned above refers to the average horizontal moving speed of the earphone unit relative to the ground within a preset time window before and after detecting the target QR code, which is used to characterize the user's walking or moving speed.
[0089] In this embodiment, once the detection module of any earphone unit successfully decodes the target QR code and triggers the detection termination signal, the main control processor immediately determines the precise time of the successful detection. Then, using this timestamp as the center, it extracts synchronous inertial sensing data within a preset time window, such as synchronous inertial sensing data within a 200ms time window (100ms before and after the timestamp). The vertical acceleration component caused by slight head rotation is removed, retaining only the horizontal acceleration data. Subsequently, the horizontal acceleration data of the inertial sensing data is numerically integrated to obtain the instantaneous velocity at each sampling moment. Finally, the arithmetic mean of all instantaneous velocities is taken to obtain the average movement speed of the earphone unit within the time window.
[0090] For example, when a user is slowly walking through an art exhibition hall and viewing the exhibits, the first earphone unit successfully detects the QR code of the exhibit next to the oil painting on the wall at 28.762 seconds. The main control processor immediately intercepts 300ms of IMU data from 28.612 seconds to 28.912 seconds, removes the vertical gravitational acceleration component, and integrates the horizontal acceleration to obtain the instantaneous velocity sequence [0.40m / s, 0.42m / s, 0.43m / s, 0.42m / s, 0.41m / s, 0.42m / s]. The average moving speed is calculated to be 0.42m / s.
[0091] Understandably, the user's movement speed directly determines the duration the target QR code remains in the camera's field of view. This embodiment predicts the remaining visible time of the QR code by obtaining the average movement speed before and after the detection time, thereby providing a reference duration for the subsequent broadcast of the item's voice information.
[0092] Step S45: Determine the duration of the target QR code in the image based on the average moving speed.
[0093] The duration of the QR code's field of view is the estimated remaining time from the moment the QR code is detected until it completely moves out of the camera's field of view.
[0094] In this embodiment, the expected duration of the target QR code in the image at the average moving speed can be calculated by looking up a table. For example, substituting the average moving speed of 0.42 m / s into the exhibition-specific mapping table, we find that 0.4 m / s corresponds to a duration of 7.2 seconds, and 0.5 m / s corresponds to a duration of 5.8 seconds. After linear interpolation, we obtain the original duration corresponding to 0.42 m / s as 6.92 seconds.
[0095] Alternatively, the duration can be obtained by calculating the time it takes for the target QR code to move to an area outside the image based on the camera pixel parameters, the pixel position of the target QR code in the image, and the direction of movement of the earphone unit.
[0096] Step S46: Determine the target voice information corresponding to the target QR code based on the duration, and output the target voice information.
[0097] In this embodiment, in addition to storing the target voice information corresponding to the target QR code, the local voice library of the earphone stores multiple sets of voice information with different durations and levels of detail for each QR code. These are typically divided into four levels: minimalist, standard, detailed, and advanced. Different target voice information is output based on the duration, thereby dynamically adjusting the output voice content according to the user's actual movement state. This avoids the voice being truncated due to excessive length when moving quickly, and also avoids the problem of insufficient information when moving slowly.
[0098] Specifically, based on the decoded content of the target QR code, a hierarchical voice information database corresponding to the QR code can be retrieved from local storage or a cloud server. Each voice message is labeled with its accurate playback duration. Then, the playback duration of each voice message is compared with the effective duration in turn. The first voice message with a playback duration less than or equal to the effective duration is selected as the target voice message and played through the speaker of the headphone unit.
[0099] For example, the QR code for the exhibit "Masterpiece B" by artist A corresponds to a tiered audio information library containing four audio versions. The advanced version states: "This painting, 'Masterpiece B,' is an oil painting created in 18xx by Post-Impressionist painter A from country C in a mental hospital in country N. It is currently housed in the Museum of Modern Art in city E. The swirling nebula, bright moon, and burning cypress trees in the painting reveal A's inner emotional world and imagination of the universe" (playback duration 8.7 seconds). The detailed version states: "Masterpiece B" is A's representative work from 18xx. The swirling nebula and burning cypress trees in the painting are highly expressive. It is currently housed in the Museum of Modern Art in city E (playback duration 5.6 seconds). The standard version states: "Masterpiece B" by A, created in 18xx, currently housed in the Museum of Modern Art in city E (playback duration 1.5 seconds). The simplified version states: "Masterpiece B" by A (playback duration 1.1 seconds). The total duration is 6.92 seconds; therefore, the 5.6-second audio information is selected for playback.
[0100] This embodiment provides an interaction method based on an earphone camera. Based on a dwell time prediction mechanism, and considering the uneven distribution of QR codes and the changing viewing positions of users in exhibition scenarios, it makes personalized predictions based on the specific location of each QR code and the user's movement direction, thereby achieving reasonable playback output of voice information.
[0101] It should be noted that all the examples above are for understanding this application only and do not constitute a limitation on the interaction method based on the headphone camera in this application. Any simple modifications based on this technical concept are within the protection scope of this application.
[0102] This application provides an earphone, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the earphone-camera-based interaction method described in the first embodiment above.
[0103] The following is for reference. Figure 5 The diagram shows a structural schematic of an earphone suitable for implementing embodiments of this application. Figure 5 The headphones shown are merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.
[0104] like Figure 5As shown, the headphones may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for headphone operation. The processing device 1001, the ROM 1002, and the RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the headset to communicate wirelessly or wiredly with other devices to exchange data. Although headsets with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.
[0105] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.
[0106] The earphone provided in this application, employing the earphone-camera-based interaction method described in the above embodiments, can solve the technical problem of random decoding when multiple QR codes appear simultaneously, leading to voice playback errors. Compared with the prior art, the beneficial effects of the earphone provided in this application are the same as those of the earphone-camera-based interaction method provided in the above embodiments, and other technical features of the earphone are the same as those disclosed in the previous embodiment method, and will not be repeated here.
[0107] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.
[0108] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0109] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the interaction method based on the headphone camera in the above embodiments.
[0110] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM, or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.
[0111] The aforementioned computer-readable storage medium may be included in the headphones; or it may exist independently and not assembled into the headphones.
[0112] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the headset, cause the headset to: acquire an image output by the camera module of the headset unit and acquire inertial sensing data output by the inertial measurement unit, wherein the inertial sensing data has the same timestamp as the image; The region priority of each preset region in the image is determined based on the inertial sensing data; According to the region priority, the QR code detection process is executed in each of the preset regions, so that the QR code detection process is terminated when the target QR code is detected. Output the voice information corresponding to the target QR code.
[0113] Computer program code for performing the operations of this application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0115] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.
[0116] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described headphone-camera-based interaction method. This solves the technical problem of random decoding when multiple QR codes appear simultaneously, leading to voice broadcast errors. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the headphone-camera-based interaction method provided in the above embodiments, and will not be repeated here.
[0117] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.
Claims
1. An interaction method based on an earphone camera, characterized in that, The earphone includes at least one earphone unit equipped with a camera module and an inertial measurement unit, and the interaction method based on the earphone camera includes: The image output by the camera module of the headphone unit is acquired, and the inertial sensing data output by the inertial measurement unit is acquired, wherein the timestamp of the inertial sensing data is the same as that of the image. The region priority of each preset region in the image is determined based on the inertial sensing data; According to the region priority, the QR code detection process is executed in each of the preset regions, so that the QR code detection process is terminated when the target QR code is detected. Output the voice information corresponding to the target QR code.
2. The interaction method based on an earphone camera as described in claim 1, characterized in that, The inertial sensing data includes triaxial acceleration and triaxial angular velocity. The step of determining the region priority of each preset region in the image based on the inertial sensing data includes: Determine the row vector composed of the triaxial acceleration and the triaxial angular velocity, and calculate the transpose vector corresponding to the row vector; Obtain the weight vector and bias of each preset region; Calculate the probability value of each preset region based on the weight vector, the bias, and the transpose vector; The priority of the region is determined based on the probability value and the preset region to which the probability value is bound.
3. The interaction method based on an earphone camera as described in claim 2, characterized in that, The step of calculating the probability value of each preset region based on the weight vector, the bias, and the transpose vector includes: Calculate the product of the weight vector and the transpose vector; The sum of the product and the bias is determined as the linear score for each of the preset regions; The probability values of each preset region are determined after the linear score is subjected to exponential operation and normalization.
4. The interaction method based on an earphone camera as described in claim 2, characterized in that, Before the steps of acquiring the image output by the camera module of the earphone unit and acquiring the inertial sensing data output by the inertial measurement unit, the interaction method based on the earphone camera further includes: Acquire the QR code image and corresponding inertial data collected by the earphone unit during the registration stage; Based on the region where the QR code is located in the QR code image, the inertial data, and the preset algorithm, the weight vector and the bias corresponding to each preset region in the headphone unit are calculated.
5. The interaction method based on an earphone camera as described in claim 1, characterized in that, The earphone unit includes a first earphone unit and a second earphone unit, the target QR code includes a first QR code collected by the first earphone unit and a second QR code collected by the second earphone unit, and the step of outputting the voice information corresponding to the target QR code includes: If there is a difference between the first QR code and the second QR code, the priority of the first QR code and the second QR code is determined based on the attribute information of the first QR code and the second QR code; The QR code with the highest priority is identified as the target QR code; Output the voice information corresponding to the target QR code.
6. The interaction method based on an earphone camera as described in claim 5, characterized in that, After the step of performing a QR code detection process in each of the preset areas according to the area priority, so as to terminate the QR code detection process when a target QR code is detected, the interaction method based on the earphone camera further includes: If the first QR code and the second QR code are the same QR code, then output the voice information corresponding to the target QR code.
7. The interaction method based on an earphone camera as described in claim 1, characterized in that, The step of performing a QR code detection process in each of the preset regions according to the region priority, so as to terminate the QR code detection process when a target QR code is detected, includes: Determine the detection parameters corresponding to the region priority, wherein the detection parameters are different for different region priorities; Based on the detection parameters and the region priority, a QR code detection process is executed for each of the preset regions, so that the QR code detection process is terminated when a target QR code is detected.
8. The interaction method based on an earphone camera as described in any one of claims 1 to 7, characterized in that, The target QR code corresponds to multiple voice messages of different durations, and the step of outputting the voice message corresponding to the target QR code includes: The average moving speed of the headphone unit is determined based on the inertial sensing data over a preset time period; The duration of the target QR code in the image is determined based on the average moving speed; Based on the duration, the target voice information corresponding to the target QR code is determined and the target voice information is output.
9. An earphone, characterized in that, The earphone includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the interaction method based on the earphone camera as described in any one of claims 1 to 8.
10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the interaction method based on the headphone camera as described in any one of claims 1 to 8.