Earphone system, earphone and control method of camera module of earphone

Abstract

The invention relates to the field of hearing equipment, in particular to an earphone system, an earphone and a control method of a camera module of the earphone. The earphone system comprises a sound acquisition module and a camera shooting module which are arranged on an earphone, and a processing module, the sound acquisition module is used for acquiring environment sound data of a space where the earphone wearer is located; the camera module is used for shooting under the control of the processing module; and the processing module is used for determining a current scene where the earphone wearer is located according to the environment sound data, and awakening the camera module to shoot under the condition that the current scene is a preset target scene.

Description

Technical Field

[0001] The present disclosure relates to the field of hearing devices, and more specifically, to an earphone system, an earphone, and a control method for a camera module of the earphone. Background Art

[0002] Headphones are a necessity in the lives of modern young people. Whether they are commuting, traveling, reading indoors, etc., young people like to wear headphones to listen to music and enjoy life. However, the headphones on the market have a single function, usually only able to play music or answer calls, and do not have the function of taking pictures. In addition, even if the headphones are equipped with a camera module, the power consumption problem of the headphones caused by the camera module needs to be solved. Summary of the invention

[0003] An object of the present disclosure is to provide an earphone system, an earphone, and a control method for a camera module of the earphone, so that the earphone can have a photo-taking function without affecting the power consumption and battery life of the earphone.

[0004] According to a first aspect of the present disclosure, there is provided an earphone system, the earphone system comprising: a sound collection module and an image capture module, and a processing module, which are arranged on the earphone;

[0005] A sound collection module, used to obtain the ambient sound data of the space where the headphone wearer is located;

[0006] A camera module, used for shooting under the control of the processing module;

[0007] The processing module is used to determine the current scene where the headphone wearer is located according to the environmental sound data, and wake up the camera module to shoot when the current scene is a preset target scene.

[0008] Optionally, determining a current scene in which the headphone wearer is located according to the ambient sound data includes:

[0009] Preprocessing the environmental sound data and extracting features of the environmental sound data;

[0010] The features are input into a pre-trained scene classification model to determine the current scene where the headphone wearer is located.

[0011] Optionally, the processing module is further used for:

[0012] The image captured by the camera module is obtained, and whether the wearer is at risk of collision is determined according to the image, and if the wearer is at risk of collision, a voice message is sent to the wearer through the earphone, wherein the voice message is used to prompt the wearer that there is a risk of collision.

[0013] Optionally, determining whether the wearer has a collision risk according to the image includes:

[0014] Identify whether a target object exists in the image, and determine the movement trajectory of the target object;

[0015] Based on the motion trajectory of the target object, determine whether there is a collision risk between the target object and the wearer.

[0016] Optionally, the processing module is further used for:

[0017] Acquire a first image and a second image from the images captured by the camera module, wherein the first image and the second image are two images acquired by the camera module at a preset time interval;

[0018] Determining a movement trajectory of the wearer based on the first image and the second image;

[0019] The determining, based on the motion trajectory of the target object, whether there is a collision risk between the target object and the wearer includes:

[0020] According to the motion trajectory of the target object and the motion trajectory of the wearer, it is determined whether there is a collision risk between the target object and the wearer.

[0021] Optionally, the earphone system further comprises a motion detection module, and the motion detection module is used to obtain inertial data of the earphone wearer;

[0022] The determining, based on the motion trajectory of the target object, whether there is a collision risk between the target object and the wearer includes:

[0023] According to the motion trajectory of the target object and the inertial data of the wearer, it is determined whether there is a collision risk between the target object and the wearer.

[0024] Optionally, the earphone system includes a first earphone body and a second earphone body, and the processing module is further used to:

[0025] Determine whether a target object exists in a first image and a second image, wherein the first image is an image taken by a camera module of the first headset body, and the second image is an image taken by a camera module of the second headset body;

[0026] respectively determining a state of each target object in the first image and the second image, wherein the state includes a stationary state and a moving state;

[0027] When the first image does not include a moving target object, the camera module of the first earphone body is controlled to sleep; when the second image does not include a moving target object, the camera module of the second earphone body is controlled to sleep.

[0028] Optionally, the processing module is further used to:

[0029] In the case where the target object in a moving state exists in any earphone body, determining an area of ​​the target object in the first image;

[0030] When the target object moves to the first area, a camera module in another earphone body is awakened to take pictures, wherein the first area is an area in the first image that overlaps with the second image.

[0031] According to a second aspect of the present disclosure, a method for controlling a camera module of an earphone is provided, the method comprising: acquiring ambient sound data of a space in which a wearer of the earphone is located; determining a current scene in which the wearer of the earphone is located based on the ambient sound data, and waking up the camera module of the earphone for shooting when the current scene is a preset target scene.

[0032] According to a third aspect of the present disclosure, there is provided an earphone, comprising the earphone system according to any one of the first aspect, or a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the steps of the method according to the second aspect are implemented.

[0033] One technical effect of the present disclosure is to provide an earphone system, including a sound collection module and a camera module, and a processing module, which are arranged on the earphone. The sound collection module can collect environmental sound data of the space where the wearer of the earphone is located, and the processing module can determine the current scene where the wearer is located based on the environmental sound data, and only wake up the camera module to shoot when the scene is a preset target scene, thereby solving the power consumption problem when the earphone camera is used.

[0034] Other features and advantages of the embodiments of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments of the present disclosure.

[0036] Figure 1 is a schematic structural diagram of an earphone system according to an embodiment;

[0037] Figure 2is a schematic diagram of two camera modules photographing an overlapping area according to an embodiment;

[0038] Figure 3 is a flow chart of a method for controlling a camera module of an earphone according to an embodiment; DETAILED DESCRIPTION

[0039] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0040] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0041] Techniques and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the techniques and equipment should be considered part of the specification.

[0042] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0043] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0044] The present application embodiment provides an earphone system 100, such as Figure 1 As shown, the earphone system 100 includes a sound collection module 101 and a camera module 102, and a processing module 103, which are arranged on the earphone; the sound collection module 101 is used to obtain the environmental sound data of the space where the earphone wearer is located; the camera module 102 is used to shoot under the control of the processing module; the processing module 103 is used to determine the current scene where the earphone wearer is located according to the environmental sound data, and when the current scene is a preset target scene, wake up the camera module to shoot.

[0045] In this example, the earphone system can be a TWS (True Wireless Stereo, true wireless stereo headphones) earphone system, or it can be a headphone system or an open headphone system. Taking the TWS earphone system as an example, it may include two earphones suitable for the left and right ears of the user, and a charging storage device. In an embodiment of the present application, the sound acquisition module and the camera module of the earphone system can be all set on the two earphone bodies, and the processing module can be set on the earphone body or in the charging storage device. In the headphone system, the sound acquisition module and the camera module can be set in a group respectively near the left and right ears to detect heart rate data respectively.

[0046] In the embodiment of the present application, the earphone body can be a small earplug or a comfortable ear hook or ear clip worn on the left and right sides of a wireless earphone; at the same time, it can also be a large earmuff in a head-mounted headset that wraps around the ears and provides excellent sound insulation, or the main part of an all-in-one earphone that integrates multiple functions and also covers the ears.

[0047] In this embodiment, the sound collection module can be in the form of a microphone or a microphone array to collect the ambient sound data of the wearer's space.

[0048] In the embodiment of the present application, the camera module may include a camera module, and the lens equipped with the camera module may be set according to actual needs. In the present embodiment, the camera module may be in a dormant state under normal circumstances to save power consumption, and will only be awakened to take images or record videos under the control of the processing module.

[0049] In this example, the processing module can obtain the ambient sound data of the wearer's environment collected by the sound collection module to determine whether to turn on the camera module. The processing module can pre-test and store the ambient sound data in various states, and when the wearer actually uses the headset, compare the collected sound data with the pre-stored data to determine the user's current scene, and turn on the camera module for shooting when the current scene is a preset target scene. The preset target scene can be a sports scene, a traffic scene, etc.

[0050] In this example, a headset system is provided, including a sound collection module and a camera module, and a processing module, which are arranged on the headset. The sound collection module can collect the ambient sound data of the space where the headset wearer is located, and the processing module can determine the current scene where the wearer is located based on the ambient sound data, and only wake up the camera module to shoot when the scene is a preset target scene, thereby solving the power consumption problem when the headset camera is used.

[0051] In an example of this embodiment, determining the current scene of the headphone wearer based on the ambient sound data includes: preprocessing the ambient sound data and extracting features of the ambient sound data; inputting the features into a pre-trained scene classification model to determine the current scene of the headphone wearer.

[0052] In this embodiment, the preprocessing of the ambient sound may include steps such as denoising, normalization and filtering to improve the accuracy of subsequent analysis. After preprocessing, the features of the ambient sound data may be extracted. In this example, the features may include at least one of time domain features, frequency domain features and time-frequency features, wherein the time domain features include energy, zero crossing rate, etc. Among them, energy is a basic feature of the sound signal in the time domain, which reflects the intensity of the signal in a certain period of time. For discrete signals, energy can be obtained by calculating the square root of the sum of the squares of the signal. The zero crossing rate refers to the number of times the signal passes through the zero point per unit time, which reflects the frequency information of the signal. For discrete signals, it can be approximated by calculating the number of symbol changes at adjacent sampling points. The frequency domain features can be extracted by short-time Fourier transform or Mel spectrum; and the time-frequency features can be combined with the time domain features and the frequency domain features, and extracted by wavelet transform, wherein the wavelet transform is a time-frequency analysis method that can provide the frequency information of the signal at different time scales. The wavelet transform uses a scalable and translational wavelet basis function to analyze the signal, which makes it more flexible in processing non-stationary signals.

[0053] In one example, after the features of the audio data are extracted, the features can also be reduced in dimension and selected by principal component analysis or linear discriminant analysis. Principal component analysis can project the original data into a new coordinate system through linear transformation, so that the variance of the projected data on the first coordinate (i.e., the first principal component) is the largest, the variance on the second coordinate (i.e., the second principal component) is the second largest, and so on. The goal is to retain the main change pattern in the data while removing noise and redundant information. By calculating the covariance matrix, eigenvalues ​​and eigenvectors of the feature matrix, the eigenvectors corresponding to the first few largest eigenvalues ​​are selected as the new coordinate axis, thereby achieving dimensionality reduction. Linear discriminant analysis is a supervised dimensionality reduction technique that aims to find a linear combination so that the projections of data between different categories on this combination can be separated as much as possible, while the data within the same category is as compact as possible. If there is already label information (such as different categories of sound), linear discriminant analysis can be used to further optimize feature selection and improve classification results. Optimizing the feature space according to the category label makes it easier for the classifier to distinguish between different categories of sound. After dimensionality reduction and selection, the pre-trained scene classification model can be used to determine the scene in which the user is located, such as indoor scene, outdoor scene, sports scene, etc.

[0054] In an example of this embodiment, an image captured by a camera module is acquired, and whether the wearer is at risk of collision is determined based on the image. If the wearer is at risk of collision, a voice message is sent to the wearer through headphones, and the voice message is used to prompt the wearer that the wearer is at risk of collision.

[0055] In this embodiment, the processing module can observe the left and right sides of the wearer based on the image captured by the camera module in the headset to identify the wearer's collision risk. Specifically, the image captured by the camera module can be used to calculate the movement speed and direction of the object in the image to determine whether there is a safety risk. In another example, a pre-trained neural network can also be used to analyze whether there is a collision risk.

[0056] In this embodiment, when there is a risk of collision, the headset can play voice information to the user through its own speaker to remind the user of the risk of collision, so that the user can avoid it and improve his own safety.

[0057] In this example, the earphone system can identify whether there is a collision risk on the left or right sides based on the images taken by the camera module, and use voice prompts when there is a risk, thereby improving the safety of the user when wearing the earphones.

[0058] In an example of this embodiment, determining whether the wearer is at risk of collision based on the image includes: identifying whether there is a target object in the image and determining the motion trajectory of the target object; and determining whether there is a risk of collision between the target object and the wearer based on the motion trajectory of the target object.

[0059] In this embodiment, the target detection algorithm can be used to identify whether there is a target object in the image. The target detection algorithm can be a pre-trained YOLO algorithm or SSD algorithm. The target object can be a movable object such as a car, a bicycle, an animal, a pedestrian, etc. Afterwards, the target object can be tracked based on a Kalman filter or a SORT (Simple Online and Realtime Tracking) algorithm to determine the target object's direction of movement, speed, and trajectory. In this way, it can be determined whether there is a risk of collision between the target object and the wearer.

[0060] In an example of the present embodiment, the processing module is also used to: obtain a first image and a second image from the image captured by the camera module, wherein the first image and the second image are two images acquired by the camera module at an interval of a preset time; determine the motion trajectory of the wearer based on the first image and the second image; determine whether there is a risk of collision between the target object and the wearer based on the motion trajectory of the target object, including: determining whether there is a risk of collision between the target object and the wearer based on the motion trajectory of the target object and the motion trajectory of the wearer.

[0061] In an embodiment of the present application, after the camera module shoots, the processing module can obtain two images in the image shot by the camera module according to a preset frame rate interval, and determine the wearer's motion trajectory based on the two images. In this example, a stationary object can be identified based on a target recognition algorithm, and then feature points can be extracted on the stationary object. By comparing the position changes of the feature points in the first image and the second image, the displacement vector of each feature point is calculated, and the motion time is calculated based on the frame rate interval of the two images, thereby determining the overall motion direction, distance and speed of the headphone wearer. Construct the wearer's motion trajectory. After determining the wearer's motion trajectory, it is possible to determine whether there is a possibility of collision between the two based on the wearer's motion trajectory and the motion trajectory of the target object. Among them, it is possible to first determine whether the two motion trajectories have an intersection. When there is no intersection, it means that there is no risk of collision between the two. When there is an intersection between the motion trajectories, it is possible to further determine the expected time for the two to reach the intersection position. If the interval between the expected time for the wearer to arrive at the position and the expected time for the target object to arrive at the position is less than a preset threshold, it means that the two may come into contact. At this time, it can be determined that there is a risk of collision between the target object and the wearer. If it is greater than or equal to the preset threshold, it means that there is no risk of collision between the two.

[0062] In an example of this embodiment, the headphone system also includes a motion detection module, which is used to obtain inertial data of the headphone wearer; based on the motion trajectory of the target object, determining whether there is a risk of collision between the target object and the wearer, including: determining whether there is a risk of collision between the target object and the wearer based on the motion trajectory of the target object and the inertial data of the wearer.

[0063] In an embodiment of the present application, the motion detection module may include an inertial measurement unit. IMU (Inertial Measurement Unit) is a device that integrates multiple sensors and is mainly used to measure and monitor the linear acceleration and angular velocity of an object. It usually includes an accelerometer and a gyroscope. The inertial measurement unit can obtain inertial data such as the speed, acceleration and direction of movement of the object. The motion detection unit can be set on the headset body.

[0064] In this example, after the inertial data of the wearer is obtained by the motion detection module, the current position of the user can be used as the initial position according to the inertial data, such as speed, acceleration and direction, and the wearer's future motion trajectory can be determined by the above inertial data. In another example, the user's motion trajectory can be further accurately determined by combining the inertial data with the image captured by the camera module. After determining the wearer's motion trajectory, it can be determined whether there is a possibility of collision between the two according to the wearer's motion trajectory and the motion trajectory of the target object, wherein it can be first determined whether there is an intersection between the two motion trajectories. If there is no intersection, it means that there is no risk of collision between the two. If there is an intersection between the motion trajectories, the expected time for the two to reach the intersection position can be further determined. If the interval between the expected time for the wearer to arrive at the position and the expected time for the target object to arrive at the position is less than a preset threshold, it means that the two may be in contact. At this time, it can be determined that there is a risk of collision between the target object and the wearer. If it is greater than or equal to the preset threshold, it means that there is no risk of collision between the two.

[0065] In this example, the wearer's own movement and trajectory can be detected through images taken by the user or the motion detection module in the headset, and then combined with the motion trajectory of the target object to more accurately predict whether there is a risk of collision between the target object and the wearer, so as to provide prompts, improve user safety, reduce false alarms, and thus improve the user experience.

[0066] In an example of this embodiment, the earphone system includes a wearing detection module arranged on the earphone, which is used to detect the wearing state of the earphone; the processing module is also used to wake up the sound collection module to obtain environmental sound data when the earphone is worn.

[0067] In this embodiment, the wearing detection module can detect whether the earphone has been worn by the user. The wearing detection module may include an optical sensor, etc. The optical sensor emits light waves (usually infrared rays). After these light waves are reflected by an object (such as an ear), they return to a receiver near the earphone, thereby measuring the distance between the earphone and the human body to perform wearing detection, thereby waking up the sound collection module to obtain environmental sound data.

[0068] In this example, the sound collection module can be enabled to collect environmental sound data only when the earphone is worn, thereby further saving the power consumption of the earphone.

[0069] In an example of this embodiment, the processing module is also used to: obtain the image taken by the camera module, and determine whether the wearer of the headset makes a preset gesture based on the image; when the wearer makes a preset gesture, control the camera module to enter a sleep state.

[0070] In this embodiment, the image captured by the camera module can be used to detect the area where a human hand appears in the image through a deep learning model, and then it is identified whether the user's gesture is a preset gesture. When the user's gesture is the preset gesture, the operation corresponding to the preset gesture is performed, for example, stopping shooting, controlling the camera module to enter a sleep state, or adjusting other operations such as aperture, focal length, and camera mode.

[0071] In an example of the present implementation, the earphone system includes a first earphone body and a second earphone body, and a processing module, which is also used to: determine whether there is a target object in a first image and a second image, wherein the first image is an image taken by a camera module of the first earphone body, and the second image is an image taken by a camera module of the second earphone body; respectively determine the state of each target object in the first image and the second image, the state including a static state and a moving state; if the first image does not contain a target object in a moving state, control the camera module of the first earphone body to sleep, and if the second image does not contain a target object in a moving state, control the camera module of the second earphone body to sleep.

[0072] In this embodiment, after waking up the camera module to shoot an image, the processing module can further determine whether there is a target object in the image shot by the camera modules of the two headphone bodies. The target detection algorithm can be used to identify whether there is a target object in the image. The target detection algorithm can be a pre-trained YOLO algorithm or SSD algorithm. In this example, the target object can be a car, a bicycle, an animal, a pedestrian, and other movable objects. Afterwards, the state of the target object can be judged according to the image. In this example, the algorithm can first identify common objects that do not move, such as tree trunks, telephone poles, buildings, etc. Afterwards, a feature reference point is determined on the stationary object, and the state of the target object is determined to be moving or stationary based on the distance between the target object and the reference point. When the image shot by any headphone body does not contain an object in a moving state, the camera module of the headphone body can be controlled to sleep to save power.

[0073] In an example of the present embodiment, the earphone system includes a first earphone body and a second earphone body, and a processing module, which is also used to: when there is a moving target object in any earphone body, determine the area of ​​the target object in the first image; when the target object moves to the first area, wake up the camera module in the other earphone body to shoot, wherein the first area is the area in the first image that overlaps with the second image.

[0074] In this embodiment, the area of ​​the image captured by the camera module can be pre-divided, specifically, it can be divided into the overlapping area of ​​the images captured by the two earphones and the independent area of ​​the image of each earphone body. Figure 2As shown, the image range captured by the two headphones of the user can be the range between the dotted lines. In this example, a wide-angle lens or other lens module can be used to capture images. The overlapping area of ​​the images captured by the two headphones can correspond to the following: Figure 2 The image is the shaded portion in the figure, and the independent area corresponds to the image of the non-shaded portion.

[0075] In this example, when one earphone body wakes up to shoot the target object and the other object is controlled to sleep, the processing module can continuously track the area of ​​the target object in the image shot by the awakened earphone body. When the target object moves to the overlapping area, the other earphone can be woken up to continue tracking and shooting the target object, so as to facilitate subsequent analysis of traffic risk status or tracking shooting. When the target object leaves the range shot by the previous earphone, if the reverse position shot by the earphone does not have a moving target object, the camera module of the earphone can be controlled to sleep based on the method of the above example.

[0076] In this example, the processing module can determine whether to turn on or unilaterally turn on the camera module based on the state of the target object to save power. At the same time, after the target object moves to the overlapping area, the dormant earphone camera module can be awakened to continue shooting to prevent the target object from leaving the field of view.

[0077] The present application also provides a method for controlling a camera module of a headset, such as Figure 3 As shown, the method includes:

[0078] Step S11, obtaining ambient sound data of the space where the headphone wearer is located;

[0079] Step S12, determining the current scene of the headphone wearer according to the ambient sound data, and waking up the camera module of the headphone to shoot when the current scene is a preset target scene.

[0080] Optionally, determining the current scene of the headphone wearer based on the ambient sound data includes: preprocessing the ambient sound data and extracting features of the ambient sound data; inputting the features into a pre-trained scene classification model to determine the current scene of the headphone wearer.

[0081] Optionally, the method also includes: acquiring images captured by the camera module, determining whether the wearer is at risk of collision based on the images, and sending a voice message to the wearer through headphones if the wearer is at risk of collision, wherein the voice message is used to remind the wearer of the risk of collision.

[0082] Optionally, determining whether the wearer is at risk of collision based on the image includes: identifying whether there is a target object in the image and determining a motion trajectory of the target object; and determining whether there is a collision risk between the target object and the wearer based on the motion trajectory of the target object.

[0083] Optionally, the method also includes: acquiring a first image and a second image from an image captured by a camera module, wherein the first image and the second image are two images acquired by the camera module at an interval of a preset time; determining a motion trajectory of the wearer based on the first image and the second image; and determining whether there is a risk of collision between the target object and the wearer based on the motion trajectory of the target object, including: determining whether there is a risk of collision between the target object and the wearer based on the motion trajectory of the target object and the motion trajectory of the wearer.

[0084] Optionally, the headset also includes a motion detection module, which is used to obtain inertial data of the headset wearer; based on the motion trajectory of the target object, determining whether there is a risk of collision between the target object and the wearer, including: determining whether there is a risk of collision between the target object and the wearer based on the motion trajectory of the target object and the inertial data of the wearer.

[0085] Optionally, the method also includes acquiring an image captured by the camera module, and determining whether the wearer of the headset makes a preset gesture based on the image; if the wearer makes a preset gesture, controlling the camera module to enter a sleep state.

[0086] Optionally, the method further includes: detecting a wearing state of the earphone, and when the earphone is worn, waking up a sound collection module to acquire ambient sound data.

[0087] Optionally, the earphone includes a first earphone body and a second earphone body, and the method further includes:

[0088] Determine whether a target object exists in a first image and a second image, wherein the first image is an image taken by a camera module of the first headset body, and the second image is an image taken by a camera module of the second headset body;

[0089] respectively determining a state of each target object in the first image and the second image, wherein the state includes a stationary state and a moving state;

[0090] When the first image does not contain a moving target object, the camera module of the first earphone body is controlled to sleep; when the second image does not contain a moving target object, the camera module of the second earphone body is controlled to sleep.

[0091] Optionally, the method further comprises:

[0092] In the case where the target object in a moving state exists in any earphone body, determining an area of ​​the target object in the first image;

[0093] When the target object moves to the first area, the camera module in the other earphone body is awakened to take pictures, wherein the first area is the area in the first image that overlaps with the second image.

[0094] An embodiment of the present application also provides an earphone, comprising any earphone system in the earphone system embodiment; or, comprising a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the steps of any method in the control method of the camera module of the earphone are implemented.

[0095] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0096] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0097] The embodiments of the present disclosure may be systems, methods and / or computer program products. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the embodiments of the present disclosure.

[0098] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0099] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0100] The computer program instructions for performing the operation of the embodiments of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, executed as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In certain embodiments, by utilizing the state information of a computer-readable program instruction to customize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit may execute a computer-readable program instruction, thereby realizing the various aspects of the embodiments of the present disclosure.

[0101] Various aspects of the embodiments of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0102] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0103] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0104] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and a part of a module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0105] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A headphone system, characterized in that: The earphone system comprises: a sound collection module and a camera module arranged on the earphone, and a processing module; A sound collection module, used to obtain the ambient sound data of the space where the headphone wearer is located; A camera module, used for shooting under the control of the processing module; The processing module is used to determine the current scene where the headphone wearer is located according to the environmental sound data, and wake up the camera module to shoot when the current scene is a preset target scene.

2. The earphone system according to claim 1, characterized in that The determining, according to the ambient sound data, the current scene in which the headphone wearer is located comprises: Preprocessing the environmental sound data and extracting features of the environmental sound data; The features are input into a pre-trained scene classification model to determine the current scene where the headphone wearer is located.

3. The earphone system according to claim 1, characterized in that The processing module is also used for: The image captured by the camera module is obtained, and whether the wearer is at risk of collision is determined according to the image, and if the wearer is at risk of collision, a voice message is sent to the wearer through the earphone, wherein the voice message is used to prompt the wearer that the wearer is at risk of collision.

4. The earphone system according to claim 3, characterized in that Determining whether the wearer has a collision risk according to the image includes: Identify whether a target object exists in the image, and determine the movement trajectory of the target object; Based on the motion trajectory of the target object, determine whether there is a collision risk between the target object and the wearer.

5. The earphone system according to claim 4, characterized in that The processing module is also used for: Acquire a first image and a second image from the images captured by the camera module, wherein the first image and the second image are two images acquired by the camera module at a preset time interval; Determining a movement trajectory of the wearer based on the first image and the second image; The determining, based on the motion trajectory of the target object, whether there is a collision risk between the target object and the wearer includes: According to the motion trajectory of the target object and the motion trajectory of the wearer, it is determined whether there is a collision risk between the target object and the wearer.

6. The earphone system according to claim 4, characterized in that The earphone system further comprises a motion detection module, wherein the motion detection module is used to obtain inertial data of the earphone wearer; The determining, based on the motion trajectory of the target object, whether there is a collision risk between the target object and the wearer includes: According to the motion trajectory of the target object and the inertial data of the wearer, it is determined whether there is a collision risk between the target object and the wearer.

7. The earphone system according to claim 1, characterized in that The earphone system comprises a first earphone body and a second earphone body, and the processing module is further used for: Determine whether a target object exists in a first image and a second image, wherein the first image is an image captured by a camera module of the first headset body, and the second image is an image captured by a camera module of the second headset body; respectively determining a state of each target object in the first image and the second image, wherein the state includes a stationary state and a moving state; When the first image does not include a moving target object, the camera module of the first earphone body is controlled to sleep; when the second image does not include a moving target object, the camera module of the second earphone body is controlled to sleep.

8. The earphone system according to claim 7, characterized in that The processing module is further used for: In the case where the target object in a moving state exists in any earphone body, determining an area of ​​the target object in the first image; When the target object moves to the first area, a camera module in another earphone body is awakened to take pictures, wherein the first area is an area in the first image that overlaps with the second image.

9. A method for controlling a camera module of an earphone, characterized in that: The method comprises: Acquiring ambient sound data of the space where the headphone wearer is located; The current scene of the headset wearer is determined according to the ambient sound data, and when the current scene is a preset target scene, the camera module of the headset is awakened to take pictures.

10. A headset, characterized in that: The invention comprises the earphone system as claimed in claims 1 to 8; or comprises a processor and a memory, wherein the memory stores computer instructions, and when the computer instructions are executed by the processor, the steps of the method as claimed in claim 9 are implemented.

Images