Earphone system, earphone and control method of camera module of earphone

Abstract

The invention relates to the field of hearing equipment, in particular to an earphone system, an earphone and a control method of a camera module of the earphone. The earphone system comprises a heart rate detection module, a camera module and a processing module, the heart rate detection module is used for acquiring heart rate data of the earphone wearer; the camera module is used for shooting under the control of the processing module; and the processing module is used for waking up the camera module to shoot according to the heart rate data.

Description

Technical Field

[0001] The present disclosure relates to the field of hearing devices, and more specifically, to a headphone system and a control method for a camera module of the headphone. Background Art

[0002] The earphones on the market have only one function, usually only able to play music or answer calls, and do not have the function of taking pictures. In addition, even if the earphones are equipped with a camera module, the power consumption problem of the earphones caused by the camera module needs to be solved. Summary of the invention

[0003] An object of the present disclosure is to provide an earphone system, an earphone, and a control method for a camera module of the earphone, so that the earphone can have a photo-taking function without affecting the power consumption and battery life of the earphone.

[0004] According to a first aspect of the present disclosure, there is provided an earphone system, the earphone system comprising: a heart rate detection module and a camera module, and a processing module, which are arranged on the earphone;

[0005] A heart rate detection module, used to obtain the heart rate data of the headphone wearer;

[0006] A camera module, used for shooting under the control of the processing module;

[0007] The processing module is used to wake up the camera module to take pictures according to the heart rate data.

[0008] Optionally, waking up the camera module to shoot according to the heart rate data includes:

[0009] When the heart rate data is greater than a first preset threshold, the camera module is awakened to take pictures.

[0010] Optionally, the processing module is further used to:

[0011] Acquire a first image and a second image from the images captured by the camera module, wherein the first image and the second image are two images acquired by the camera module at a preset time interval;

[0012] determining a motion state of the headset based on the first image and the second image;

[0013] When the motion state of the earphone does not meet a preset condition, the camera module is controlled to sleep.

[0014] Optionally, the earphone system comprises a first earphone body worn on the left ear of the user and a second earphone body worn on the right ear of the user, and the processing module is further used to:

[0015] Determine an overlapping portion of a first image and a second image, wherein the first image is an image captured by a camera module of the first earphone body, and the second image is an image captured by a camera module of the second earphone body;

[0016] It is determined whether there is a target object in the overlapping portion, and when there is no target object in the overlapping portion, the camera modules of the first earphone body and the second earphone body are controlled to sleep.

[0017] Optionally, the earphone system further includes a wireless communication module;

[0018] The step of waking up the camera module to shoot according to the heart rate data includes:

[0019] When the heart rate data is greater than a first preset threshold or less than a second preset threshold, waking up the camera module to take pictures;

[0020] The processing module is also used to send the image captured by the camera module to the target device through the wireless communication module.

[0021] Optionally, the earphone system further comprises a speaker module arranged on the earphone;

[0022] The processing module is also used to send a sound prompt to the wearer of the headset through the speaker module when the heart rate data is greater than a first preset threshold or the heart rate data is less than a second preset threshold, and the sound prompt is used to prompt the wearer that there is an abnormal heart rate.

[0023] Optionally, the processing module is further used for:

[0024] Determining the head posture of the headset wearer according to the image captured by the camera module;

[0025] Determine whether to send the image captured by the camera module to the target device through the wireless communication module according to the head posture.

[0026] Optionally, the processing module is further used to:

[0027] Acquire the image captured by the camera module, and determine whether the wearer of the headset makes a preset gesture according to the image;

[0028] When the wearer makes a preset gesture, the camera module is controlled to enter a dormant state.

[0029] Optionally, waking up the camera module to shoot according to the heart rate data includes:

[0030] The heart rate data is input into a pre-trained heart rate recognition model, and when the output result of the model meets the preset conditions, the camera module is controlled to shoot.

[0031] Optionally, the earphone system includes a wearing detection module arranged on the earphone, for detecting a wearing state of the earphone;

[0032] The processing module is further configured to wake up the heart rate detection module to obtain the heart rate data of the wearer when the earphone is worn.

[0033] According to a second aspect of the present disclosure, a method for controlling a camera module of an earphone is provided, the method comprising: acquiring heart rate data of a wearer of the earphone; and waking up the camera module of the earphone for shooting according to the heart rate data.

[0034] According to a third aspect of the present disclosure, there is provided an earphone, comprising the earphone system according to any one of the first aspect, or a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the steps of the method according to the second aspect are implemented.

[0035] One technical effect of the present disclosure is that it provides an earphone system, including a heart rate detection module and a camera module, and a processing module, which are arranged on the earphone. Among them, the heart rate detection module can obtain the heart rate data of the earphone wearer, and the processing module can wake up the camera module for shooting based on the heart rate data of the earphone wearer. In this way, the camera module of the earphone can be intelligently awakened based on the heart rate data, thereby avoiding power consumption problems. At the same time, for sports scenes, when it is inconvenient for users to take pictures by themselves, they do not have to worry about not being able to record the sports process, and can automatically shoot through the camera module in the earphone.

[0036] Other features and advantages of the embodiments of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments of the present disclosure.

[0038] Figure 1 is a schematic structural diagram of an earphone system according to an embodiment;

[0039] Figure 2 is a schematic diagram of an overlapped portion according to an embodiment;

[0040] Figure 3 It is a flowchart of a method for controlling a camera module of an earphone according to one embodiment. DETAILED DESCRIPTION

[0041] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0042] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0043] Techniques and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the techniques and equipment should be considered part of the specification.

[0044] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0045] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0046] The present application embodiment provides an earphone system 100, such as Figure 1 As shown, the earphone system 100 includes a heart rate detection module 101 and a camera module 102, and a processing module 103, which are arranged on the earphone; the heart rate detection module 101 is used to obtain the heart rate data of the earphone wearer; the camera module 102 is used to take pictures under the control of the processing module; the processing module 103 is used to wake up the camera module for taking pictures according to the heart rate data.

[0047] In this example, the earphone system can be a TWS (True Wireless Stereo) earphone system or a headphone system. Taking the TWS earphone system as an example, it may include two earphone bodies suitable for the left and right ears of the user, and a charging storage device. In an embodiment of the present application, the heart rate detection module and the camera module of the earphone system can be all set on the two earphone bodies, and the processing module can be set on the earphone body or in the charging storage device. In the headphone system, the heart rate detection module and the camera module can be set in a group respectively near the left and right ears to detect the heart rate data respectively.

[0048] In the embodiment of the present application, the heart rate detection module can use a photoelectric heart rate sensor to detect the change of blood flow through an LED light source and a photosensitive sensor to detect the user's heart rate. In another example, the heart rate detection module can also include a microphone on the headset, and the microphone obtains the wearer's heart rate information through the sound or vibration generated by the heartbeat.

[0049] In the embodiment of the present application, the camera module also refers to the headset, and the camera module may include a camera module, and the lens equipped with the camera module may be set according to actual needs. In this embodiment, the camera module can be in a dormant state under normal circumstances to save power consumption, and will only be awakened to take images or record videos under the control of the processing module.

[0050] In this example, the processing module can obtain the wearer's heart rate data collected by the heart rate detection module to determine whether to start the camera module. The processing module can pre-test and store the heart rate data in the relevant motion state, and when the wearer actually uses the headset, the collected heart rate data is compared with the pre-stored heart rate data to determine whether the user is currently in motion. In this way, the camera module of the headset is controlled to wake up and shoot to obtain images of motion scenes.

[0051] In this example, a headset system is provided, including a heart rate detection module and a camera module, and a processing module, which are arranged on the headset. The heart rate detection module can obtain the heart rate data of the headset wearer, and the processing module can wake up the camera module for shooting based on the heart rate data of the headset wearer. In this way, the headset can be intelligently awakened based on the heart rate data, thereby avoiding power consumption problems. At the same time, for sports scenes, if it is inconvenient for users to take pictures by themselves, they do not have to worry about not being able to record the sports process, and can automatically shoot through the camera module in the headset.

[0052] In an example of this embodiment, the earphone system includes a wearing detection module arranged on the earphone, which is used to detect the wearing status of the earphone; the processing module is also used to wake up the heart rate detection module to obtain the wearer's heart rate data when the earphone is worn.

[0053] In this embodiment, the wearing detection module can detect whether the earphone is worn by the user. The wearing detection module may include an optical sensor, etc. The optical sensor emits light waves (usually infrared rays). After these light waves are reflected by an object (such as an ear), they return to a receiver near the earphone, thereby measuring the distance between the earphone and the human body to perform wearing detection, thereby waking up the heart rate detection module to obtain the heart rate data of the earphone.

[0054] In this example, the heart rate detection module can be enabled to detect the heart rate data of the headset only when the headset is worn, thereby further saving the power consumption of the headset.

[0055] In an example of this embodiment, waking up the camera module for shooting according to the heart rate data includes: waking up the camera module for shooting when the heart rate data is greater than a first preset threshold.

[0056] In this example, the heart rate data may include the user's heart rate value. When the heart rate data is greater than a first preset threshold, it can be determined that the user is in motion or in a state with a high heart rate. At this time, the processing module can control the camera module to wake up and shoot, and record images or videos in the motion state; or, the surrounding environment when the heart rate is high.

[0057] In an example of the present embodiment, the processing module is also used to: obtain a first image and a second image from the image taken by the camera module, wherein the first image and the second image are two images obtained by the camera module at an interval of a preset time; determine the motion state of the earphone based on the first image and the second image; and control the camera module to sleep when the motion state of the earphone does not meet the preset conditions.

[0058] In this embodiment, the processing module can obtain two images at preset time intervals in the image taken by the camera module, for example, two images are obtained in the image at an interval of 100ms, and then the motion state of the headset is determined according to the content of the two images. Specifically, the position of certain specific objects in the image, specifically the position change of the same object in the two images, can be identified through an image analysis algorithm to determine the motion state of the headset, such as the motion speed and direction. After determining the motion state of the headset, it can be determined whether the motion state of the headset meets the preset conditions, such as the preset conditions that the motion speed is less than a preset threshold or the motion direction is consistent with the stored head-turning motion direction. When the motion state meets the preset conditions, it means that the wearer of the headset is not in motion, and the processing module can control the camera module to sleep. Avoid erroneous triggering of the camera module to work due to erroneous heart rate data, and further reduce the overall power consumption of the headset.

[0059] In an example of the present embodiment, the earphone system includes a first earphone body worn on the left ear of the user and a second earphone body worn on the right ear of the user, and the processing module is further used to: determine the overlapping part of the first image and the second image, wherein the first image is an image taken by the shooting module of the first earphone body, and the second image is an image taken by the shooting module of the second earphone body; determine whether there is a target object in the overlapping part, and if there is no target object in the overlapping part, control the camera modules of the first earphone body and the second earphone body to sleep.

[0060] In the embodiment of the present application, the earphone body can be a small earplug or a comfortable ear hook or ear clip worn on the left and right sides of a wireless earphone; at the same time, it can also be a large earmuff in a head-mounted headset that wraps around the ears and provides excellent sound insulation, or the main part of an all-in-one earphone that integrates multiple functions and also covers the ears.

[0061] In an example of this embodiment, the processing module may determine the overlapped portion of the images captured by the two earphones based on the images captured by the two earphones in combination with an image recognition algorithm or other algorithms, such as Figure 2 As shown, the image range captured by the two headphones of the user can be the range between the dotted lines. In this example, a wide-angle lens or other lens module can be used to capture images. The overlapping part of the images captured by the two headphones can be as shown in FIG. Figure 2 The shadow in the figure is the part directly in front of the user's head, which is also the part that the user usually pays attention to. After that, it is detected whether there is a target object in the overlapping part, which can be determined by a pre-trained target detection model. For example, the target object can be a display device such as a mobile phone, a tablet computer or a monitor, or a ball object such as a football or a basketball. In the case that there is no target object, the camera module can be controlled to sleep, and in the case that there is a target object, the camera module can be controlled to continue recording.

[0062] In this embodiment, when a user watches a film or TV show, plays a game, or watches various ball games, the user's heart rate usually increases when watching exciting clips. Therefore, the processing module can turn on the camera to shoot when the user's heart rate is high for the above scenarios, and confirm whether there are target objects such as displays or balls based on the content shot by the camera, so as to avoid other situations with high heart rates that may cause the camera module to be falsely triggered, resulting in energy waste of the headphones.

[0063] In an example of this embodiment, the earphone system also includes a wireless communication module; the camera module is awakened for shooting according to the heart rate data, including: when the heart rate data is greater than a first preset threshold or less than a second preset threshold, the camera module is awakened for shooting; the processing module is also used to send the image taken by the camera module to the target device via the wireless communication module.

[0064] In this embodiment, the headset system may also include a wireless communication module, such as a Bluetooth module or a WiFi module, etc. The module may be provided on the headset and the storage box of the headset, and be wirelessly connected to the terminal device. In this example, the target device may be a communication device of the headset wearer directly connected to the headset system, or a communication device of an emergency contact of the headset wearer, or a terminal device of a medical system, etc. In this example, the processing module of the headset system may send the image captured by the camera module directly to the target device through the wireless communication module, or may upload the image to the cloud through the wireless communication module, and transfer the image to the target device through the cloud.

[0065] In this embodiment, the processing module can wake up the camera module to shoot only when the heart rate is greater than the first preset threshold or less than the second preset threshold. The first preset threshold in this example can be a higher threshold. When the user's heart rate is greater than the threshold, it proves that the user's heart function is abnormal and may be dangerous. At this time, the user's current state and environment can be sent to the target device through the camera module for alarm. Similarly, the second preset threshold can be a relatively low threshold. When the user's heart rate is less than the threshold, the user's current state and environment can also be sent to the target device through the camera module for alarm.

[0066] In this example, the earphone system can monitor the user's heart rate and determine whether to wake up the camera module for shooting based on the relationship between the user's heart rate and a preset threshold. When the wearer's heart function is abnormal, the system can obtain the condition of the user and the surrounding environment and send it to emergency contacts or medical systems, so that other personnel can obtain the wearer's location and condition and deal with it as soon as possible.

[0067] In an example of this embodiment, the earphone system also includes a speaker module arranged on the earphone; the processing module is also used to send a sound prompt to the wearer of the earphone through the speaker module when the heart rate data is greater than a first preset threshold or the heart rate data is less than a second preset threshold, and the sound prompt is used to prompt the wearer that there is an abnormal heart rate.

[0068] In this embodiment, when the heart rate data of the wearer of the headset is greater than the first preset threshold or less than the second preset threshold, the speaker module of the headset can also issue a corresponding voice prompt, for example, a voice message such as "heart rate is too high" or "heart rate is too low". In this way, the user can be prompted with the heart rate status of the wearer of the headset, which is convenient for the user to determine his or her own status.

[0069] In an example of this embodiment, the processing module is also used to: determine the head posture of the headphone wearer based on the image captured by the camera module; and determine whether to send the image captured by the camera module to the target device through the wireless communication module based on the head posture.

[0070] In this embodiment, since the position of the headset synchronized with the user is fixed when the user wears the headset, the processing module can determine the position of the headset based on the image captured by the headset, thereby further determining the position of the user's head. In this example, the processing module can identify the current position of the headset based on a specific algorithm. For example, it can be determined based on an algorithm of feature points. Specifically, the feature points in the image can be first extracted and matched. Based on the matched feature point pairs, the basic matrix or the essential matrix is ​​used, and then the rotation matrix and translation vector of the headset are solved in combination with the intrinsic matrix of the camera module, thereby further determining the position of the user's head. Or in certain specific scenarios, such as when shooting roads or buildings, the vanishing point in the image can be used to estimate the pitch angle and yaw angle of the camera module. The vanishing point is the point where parallel lines intersect in the image. By calculating the position of the vanishing point, some posture information of the headset can be derived, thereby determining the posture information of the user's head.

[0071] In this embodiment, it can be determined whether to send the image captured by the camera module to the target device through the wireless communication module according to the user's head posture. For example, when the user's head posture is in a level or normal head-up state, it means that the user's current state is not urgent, or the data of the heart rate sensor may be abnormal, and the image may not be sent to the target device through the wireless communication module. When the user's head posture is in a posture of lowering the head, tilting the head, or lying flat, it means that the user's current state is more urgent. At this time, the image captured by the camera module can be directly sent to the target device.

[0072] In an example of this embodiment, the processing module is also used to: obtain the image taken by the camera module, and determine whether the wearer of the headset makes a preset gesture based on the image; when the wearer makes a preset gesture, control the camera module to enter a sleep state.

[0073] In this embodiment, the image captured by the camera module can be used to detect the area where a human hand appears in the image through a deep learning model, and then it is identified whether the user's gesture is a preset gesture. When the user's gesture is the preset gesture, the operation corresponding to the preset gesture is performed, for example, stopping shooting, controlling the camera module to enter a sleep state, or adjusting other operations such as aperture, focal length, and camera mode.

[0074] In an example of this embodiment, waking up the camera module for shooting according to the heart rate data includes: inputting the heart rate data into a pre-trained heart rate recognition model, and controlling the camera module to shoot when the model output result meets the preset conditions.

[0075] In this embodiment, the heart rate data may also include an ECG (electrocardiogram) signal. In addition to determining whether to turn on the camera module based on the heart rate value, the ECG signal may also be input into a pre-trained recognition model. Specifically, the collected ECG signal may be pre-processed first, for example, by bandpass filtering, and the original ECG signal may be bandpass filtered to filter out high-frequency noise and low-frequency baseline drift; differential filtering: then, differential filtering is performed to enhance the slope characteristics of the R wave while suppressing the P wave and T wave; squaring: the signal after differential filtering is squared to ensure that all signal amplitudes are positive to avoid detection difficulties caused by downward peaks.

[0076] After that, the R peak position is obtained by sliding integration to further highlight the R wave characteristics, and then a dual threshold detection method is used, in which the second threshold is half of the first threshold. If the peak is greater than the first threshold, it is regarded as an R wave; if this condition is not met, but no R wave appears within 1.5 times the average RR interval, a "retrospective re-judgment" is performed, and the peak value is compared with the second threshold. If the condition is met and the R wave position rule is met, it is determined to be an R wave. After that, the heart rate variability is analyzed by calculating the time interval (RR interval) between adjacent R waves to analyze the heart rate variability. Heart rate variability reflects the balance of sympathetic and parasympathetic nerve activity. Then, a pre-trained heart rate recognition model (such as support vector machine SVM, convolutional neural network CNN) is used to identify abnormal heart rate patterns. When the output result is an abnormal heart rate, the processing module can control the camera module to wake up and shoot.

[0077] The present application also provides a method for controlling a camera module of a headset, such as Figure 3 As shown, it includes: step S11, obtaining the heart rate data of the headphone wearer;

[0078] Step S12, waking up the camera module of the headset to take pictures according to the heart rate data.

[0079] Optionally, waking up the camera module of the headset for shooting according to the heart rate data includes: waking up the camera module of the headset for shooting when the heart rate data is greater than a first preset threshold.

[0080] Optionally, the method also includes: acquiring a first image and a second image from the image captured by the camera module, wherein the first image and the second image are two images acquired by the camera module at an interval of a preset time; determining the motion state of the earphone based on the first image and the second image; and controlling the camera module to sleep when the motion state of the earphone does not meet preset conditions.

[0081] Optionally, the earphone system includes a first earphone body worn on the left ear of the user and a second earphone body worn on the right ear of the user, and the method further includes: determining an overlapping portion of a first image and a second image, wherein the first image is an image captured by a camera module of the first earphone body, and the second image is an image captured by a camera module of the second earphone body;

[0082] Determine whether there is a target object in the overlapping part, and when there is no target object in the overlapping part, control the camera modules of the first earphone body and the second earphone body to sleep.

[0083] Optionally, waking up the camera module of the headset for shooting according to the heart rate data includes: waking up the camera module for shooting when the heart rate data is greater than a first preset threshold or less than a second preset threshold; the method also includes: sending the image taken by the camera module to the target device via wireless communication.

[0084] Optionally, the method also includes sending a sound prompt to the wearer of the headset through a speaker of the headset when the heart rate data is greater than a first preset threshold or the heart rate data is less than a second preset threshold, and the sound prompt is used to prompt the wearer that there is an abnormal heart rate.

[0085] Optionally, the method further includes: determining a head posture of the headphone wearer according to an image captured by the camera module; and determining whether to send the image captured by the camera module to a target device according to the head posture.

[0086] Optionally, the method also includes: acquiring an image captured by the camera module, and determining whether the wearer of the headset makes a preset gesture based on the image; if the wearer makes a preset gesture, controlling the camera module to enter a sleep state.

[0087] Optionally, waking up the camera module of the headset for shooting according to the heart rate data includes: inputting the heart rate data into a pre-trained heart rate recognition model, and waking up the camera module for shooting when the model output result meets preset conditions.

[0088] Optionally, obtaining the heart rate data of the headphone wearer includes: when the headphone is worn, waking up a heart rate detection module in the headphone to obtain the heart rate data of the wearer.

[0089] An embodiment of the present application also provides an earphone, comprising any earphone system in the earphone system embodiment; or, comprising a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the steps of any method in the control method of the camera module of the earphone are implemented.

[0090] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0091] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0092] The embodiments of the present disclosure may be systems, methods and / or computer program products. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the embodiments of the present disclosure.

[0093] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0094] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0095] The computer program instructions for performing the operation of the embodiments of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of a computer-readable program instruction, and the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the embodiments of the present disclosure.

[0096] Various aspects of the embodiments of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0097] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0098] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0099] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and a part of a module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0100] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A headphone system, characterized in that: The earphone system comprises: a heart rate detection module and a camera module arranged on the earphone, and a processing module; A heart rate detection module, used to obtain the heart rate data of the headphone wearer; A camera module, used for shooting under the control of the processing module; The processing module is used to wake up the camera module to take pictures according to the heart rate data.

2. The earphone system according to claim 1, characterized in that The step of waking up the camera module to shoot according to the heart rate data includes: When the heart rate data is greater than a first preset threshold, the camera module is awakened to take pictures.

3. The earphone system according to claim 2, characterized in that The processing module is further used for: Acquire a first image and a second image from the images captured by the camera module, wherein the first image and the second image are two images acquired by the camera module at a preset time interval; determining a motion state of the headset based on the first image and the second image; When the motion state of the earphone does not meet a preset condition, the camera module is controlled to sleep.

4. The earphone system according to claim 2, characterized in that The earphone system comprises a first earphone body worn on the left ear of a user and a second earphone body worn on the right ear of the user, and the processing module is further used for: Determine an overlapping portion of a first image and a second image, wherein the first image is an image captured by a camera module of the first earphone body, and the second image is an image captured by a camera module of the second earphone body; It is determined whether there is a target object in the overlapping portion, and when there is no target object in the overlapping portion, the camera modules of the first earphone body and the second earphone body are controlled to sleep.

5. The earphone system according to claim 1, characterized in that The headset system also includes a wireless communication module; The step of waking up the camera module to shoot according to the heart rate data includes: When the heart rate data is greater than a first preset threshold or less than a second preset threshold, waking up the camera module to take pictures; The processing module is also used to send the image captured by the camera module to the target device through the wireless communication module.

6. The earphone system according to claim 5, characterized in that The earphone system further comprises a speaker module arranged on the earphone; The processing module is also used to send a sound prompt to the wearer of the headset through the speaker module when the heart rate data is greater than a first preset threshold or the heart rate data is less than a second preset threshold, and the sound prompt is used to prompt the wearer that there is an abnormal heart rate.

7. The earphone system according to claim 5, characterized in that The processing module is also used for: Determining the head posture of the headset wearer according to the image captured by the camera module; Determine whether to send the image captured by the camera module to the target device through the wireless communication module according to the head posture.

8. The earphone system according to claim 1, characterized in that The processing module is further used for: Acquire the image captured by the camera module, and determine whether the wearer of the headset makes a preset gesture according to the image; When the wearer makes a preset gesture, the camera module is controlled to enter a dormant state.

9. The earphone system according to claim 1, characterized in that The step of waking up the camera module to shoot according to the heart rate data includes: The heart rate data is input into a pre-trained heart rate recognition model, and when the output result of the model meets the preset conditions, the camera module is awakened to take pictures.

10. The earphone system according to claim 1, characterized in that The earphone system comprises a wearing detection module arranged on the earphone, for detecting the wearing state of the earphone; The processing module is further configured to wake up the heart rate detection module to obtain the heart rate data of the wearer when the earphone is worn.

11. A method for controlling a camera module of an earphone, characterized in that: The method comprises: Acquiring heart rate data of the headphone wearer; The camera module of the headset is awakened according to the heart rate data to take pictures.

12. A headset, characterized in that: The invention comprises the earphone system as claimed in claims 1 to 10; or comprises a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the steps of the method as claimed in claim 11 are implemented.

Images