Shooting method, wearable device and readable storage medium

Abstract

The invention relates to the technical field of wearable equipment, and provides a shooting method, wearable equipment and a computer readable storage medium, the user state change condition is monitored based on user motion parameters, and important events in shot images are monitored based on event importance parameters. And monitoring the change amplitude of the shooting scene based on the scene change parameter, namely obtaining the target priority from multiple dimensions such as the user motion parameter, the event importance score and the scene change parameter, and dynamically adjusting the shooting parameter of the shooting device in the wearable equipment based on the target priority. Therefore, the wearable device can adaptively adjust the shooting parameters in different motions and scenes, recording quality of key events by the wearable device is guaranteed, recording redundant data volume is reduced, energy consumption of the wearable device during shooting of non-key contents is reduced, and endurance time of the wearable device is prolonged.

Description

Technical Field

[0001] The present invention relates to the technical field of wearable devices, and in particular to a photographing method, a wearable device and a computer-readable storage medium. Background Art

[0002] With the increasing popularity of wearable devices, users have a growing need to take images or videos anytime, anywhere. However, long-term high-frame-rate, high-resolution recording puts serious power consumption pressure on wearable video recording devices. Therefore, how to improve the battery life of wearable video recording devices while ensuring the recording effect is an urgent problem that wearable video recording devices need to solve. Summary of the invention

[0003] The main purpose of the present invention is to provide a shooting method, a wearable device and a computer-readable storage medium, aiming to improve the battery life of the wearable recording device.

[0004] To achieve the above object, the present invention provides a shooting method, which comprises the following steps: Obtaining user motion parameters of the wearable device; Acquire a scene change parameter of the environment in which the wearable device is located, and acquire an event importance parameter of the wearable device; Calculating a target priority according to the user motion parameter, the scene change parameter and the event importance parameter; The shooting parameters of the shooting device of the wearable device are adjusted according to the target priority, and shooting is performed according to the adjusted shooting parameters.

[0005] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a processor, a memory, and a shooting program stored in the memory and executable by the processor, wherein when the shooting program is executed by the processor, the steps of the shooting method as described above are implemented.

[0006] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a shooting program is stored, wherein when the shooting program is executed by a processor, the steps of the shooting method as described above are implemented.

[0007] The present application provides a shooting method, which is applied to a wearable device. The method obtains the user motion parameters of the wearable device; obtains the scene change parameters of the environment in which the wearable device is located, and obtains the event importance parameters of the wearable device; calculates the target priority according to the user motion parameters, the scene change parameters and the event importance parameters; adjusts the shooting parameters of the shooting device of the wearable device according to the target priority, and shoots according to the adjusted shooting parameters. In the above manner, the present application monitors the user state change based on the user motion parameters, monitors the important events in the captured image based on the event importance parameters, and monitors the change amplitude of the captured scene based on the scene change parameters, that is, the target priority is obtained from multiple dimensions such as the user motion parameters, the event importance score and the scene change parameters, and the shooting parameters of the shooting device in the wearable device are dynamically adjusted based on the target priority. As a result, the wearable device can adaptively adjust the shooting parameters in different motions and scenes, while ensuring the quality of recording of key events by the wearable device, reducing the amount of redundant data recorded, reducing the energy consumption of the wearable device when shooting non-critical content, thereby improving the battery life of the wearable device. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 It is a flowchart of a first embodiment of a photographing method of the present invention; Figure 2 It is a flowchart of a second embodiment of a photographing method of the present invention; Figure 3 is a schematic diagram of a flow chart of a third embodiment of a photographing method of the present invention; Figure 4 The figure is a schematic diagram of the hardware structure of the wearable device involved in the embodiment of the present invention.

[0009] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0010] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0011] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0012] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0013] It should be further understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0014] The shooting method involved in the embodiment of the present invention is mainly applied to a wearable device, which may be smart glasses, a head-mounted display or other wearable devices with a shooting function.

[0015] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0016] Reference Figure 1 , Figure 1 FIG. 1 is a flow chart of a first embodiment of a photographing method according to the present invention.

[0017] like Figure 1 As shown, an embodiment of the present invention provides a shooting method, which is applied to a wearable device, wherein the wearable device includes: a motion sensor for collecting the user's body movement amplitude; an eye movement sensor for collecting the user's eye movement data, such as the eye movement amplitude; a shooting device, such as a camera sensor, for shooting videos or images; a processing unit, including a low-power real-time processor and a high-performance processor, for periodically analyzing the data collected by the motion sensor, the eye movement sensor, and the shooting device. A biosensor may also be included to collect the user's biological parameters such as heart rate, blood pressure, and body temperature. The following is an example of smart glasses.

[0018] In one embodiment, the photographing method includes steps S10 to S40.

[0019] Step S10, obtaining user motion parameters of the wearable device; In this embodiment, the user motion parameters obtained can be collected in real time by the sensor, that is, reading the user motion parameters of the user at the current moment, or can be collected by the sensor in a preset period, that is, reading the user motion parameters collected by the sensor within the preset period. The user motion parameters include user body motion amplitude parameters and user eye movement amplitude parameters. When the body motion amplitude data S1 and the eye movement amplitude data S2 are read, the body motion amplitude data S1 and the eye movement amplitude data S2 are filtered and normalized, and the body motion amplitude data S1 and the user eye movement amplitude data S2 are normalized to [0,10]. Among them, the closer the body motion amplitude data S1 is to 10, the more intense the user's body movement is. The closer the eye movement amplitude data S2 is to 10, the more frequent the user's line of sight moves or the focus shifts.

[0020] Step S20, obtaining a scene change parameter of the environment in which the wearable device is located, and obtaining an event importance parameter of the wearable device; In this embodiment, the scene change parameter S3 may be the scene change amplitude of the environment (such as the shooting scene) where the smart glasses are located at the current moment and the previous moment. It may also be the scene change amplitude of the environment where the smart glasses are located within a preset period. The scene change parameter S3 includes light intensity change and temperature change, etc.

[0021] Furthermore, the light change parameters of the environment in which the wearable device is located can be detected by optical flow or image difference method, or the ambient light change can be detected by a photosensitive sensor to obtain the light intensity change amplitude, and the temperature change parameters of the environment in which the wearable device is located can be detected by a temperature sensor to obtain the temperature change amplitude.

[0022] The event importance parameter S4 may be an event importance parameter corresponding to an image captured by the smart glasses at the current moment, or may be an event importance parameter corresponding to multiple images captured by the smart glasses within a preset period.

[0023] After obtaining the scene change parameter S3 and the event importance parameter S4, the scene change parameter S3 and the event importance parameter S4 are normalized to [0,100]. The closer the scene change parameter S3 is to 100, the greater the scene change amplitude of the environment in which the smart glasses are located. The closer the event importance parameter S4 is to 100, the higher the importance of the key event in the captured image at the current moment or within a preset period.

[0024] Furthermore, the shooting method further includes: Obtaining parameter collection periods corresponding to the user motion parameter, the scene change parameter, and the event importance parameter respectively; Based on each parameter collection cycle, the user motion parameter, the scene change parameter and the event importance parameter are collected or updated respectively.

[0025] In this embodiment, in order to further balance the shooting quality of key events and reduce the power consumption of the device, different parameter acquisition cycles are set for each parameter, and different sensors are set with different cycle frequencies (i.e., working cycles) based on each parameter acquisition cycle. The higher the cycle frequency, the shorter the working cycle, the faster the data update speed, the faster the response to the adjustment event of the shooting parameters, and the greater the power consumption of the device.

[0026] In a specific embodiment, the parameter collection period can also be determined according to the data reading frequency. The higher the data reading frequency, the higher the data collection frequency of the sensor and the shorter the parameter collection period.

[0027] The parameter collection period of the event importance parameters can be increased when the collection period of any one of the user motion parameters or scene change parameters is shortened, or the analysis frequency of the event importance parameters can be increased within a preset time period after the key event is identified, and the collection period of the event importance parameters can be shortened to shorten the response time of the key event.

[0028] In more embodiments, a fixed period may be used as the collection period. For example: Perform high-frequency data collection on S1 and S2, such as a parameter collection cycle of every second, a few seconds, or tens of seconds, etc., to provide the latest data for the calculation of priority or event importance parameters in real time; Perform medium frequency acquisition on S3, such as a parameter acquisition cycle of several minutes or more than ten minutes, for periodic scene change analysis and periodic correction or update of S3.

[0029] S4 is collected at a low frequency, such as a parameter collection period of 30 minutes, 1 hour or 2 hours, etc., to score the importance of events based on S1, S2 and S3, and key frames can be further identified based on the scoring results.

[0030] Step S30, calculating the target priority according to the user motion parameter, the scene change parameter and the event importance parameter; In this embodiment, the values ​​corresponding to the user motion parameter, the scene change parameter, and the event importance parameter can be substituted into the priority calculation formula to calculate the target priority. The higher the priority, the higher the importance of the image required to be captured by the smart glasses, and the shooting parameters of the shooting device of the smart glasses need to be improved, such as increasing the frame rate and resolution, so as to improve the image quality of the captured image; the lower the priority, the lower the importance of the image required to be captured by the smart glasses, and the shooting device can be controlled to enter a low power consumption state, such as reducing the frame rate and resolution, so as to reduce the power consumption of the shooting device.

[0031] Step S40: adjusting the shooting parameters of the shooting device of the wearable device according to the target priority, and shooting according to the adjusted shooting parameters.

[0032] In this embodiment, the shooting parameters include basic image parameters such as frame rate and resolution, and also include environmental adaptation parameters such as exposure control and anti-shake parameters, and user interaction parameters such as shooting model and focus parameters.

[0033] The shooting parameter can be adjusted by comparing the target priority with the average priority (or basic priority). If the target priority is less than the average, the shooting parameter is reduced, and the greater the difference, the greater the reduction; if the target priority is greater than the average, the shooting parameter is increased, and the greater the difference, the greater the increase. The shooting parameter can also be adjusted based on the target priority, and the shooting parameter is increased or decreased to the target shooting parameter. The shooting device is controlled to shoot with the target shooting parameter.

[0034] This embodiment provides a shooting method applied to a wearable device, the method obtains the user motion parameters of the wearable device; obtains the scene change parameters of the environment in which the wearable device is located, and obtains the event importance parameters of the wearable device; calculates the target priority according to the user motion parameters, the scene change parameters and the event importance parameters; adjusts the shooting parameters of the shooting device of the wearable device according to the target priority, and shoots according to the adjusted shooting parameters. In the above manner, the present application monitors the user state change based on the user motion parameters, monitors the important events in the captured image based on the event importance parameters, and monitors the change amplitude of the captured scene based on the scene change parameters, that is, the target priority is obtained from multiple dimensions such as the user motion parameters, the event importance score and the scene change parameters, and the shooting parameters of the shooting device in the wearable device are dynamically adjusted based on the target priority. As a result, the wearable device can adaptively adjust the shooting parameters under different motions and scenes, while ensuring the recording quality of the wearable device for key events, reducing the amount of redundant data recorded, reducing the energy consumption of the wearable device when shooting non-critical content, thereby improving the battery life of the wearable device.

[0035] Reference Figure 2 , Figure 2 FIG. 4 is a flow chart of a second embodiment of a photographing method according to the present invention.

[0036] In this embodiment, the step S30 includes: Step S31, obtaining the scene type of the environment in which the wearable device is located, and determining the adjustment coefficients corresponding to the user motion parameter, the event importance parameter, and the scene change parameter respectively based on the scene type; Step S32, calculating the target priority based on each adjustment coefficient, the user motion parameter, the event importance parameter and the scene change parameter.

[0037] The step of calculating the target priority according to the user motion parameter, the scene change parameter and the event importance parameter includes: An index parameter of the event importance parameter is obtained, and the target priority is calculated based on the user motion parameter, the scene change parameter, the event importance parameter and the index parameter.

[0038] In this embodiment, different parameters have different importance in different shooting scenes. For example, in medical monitoring scenes and public safety monitoring scenes, user motion parameters are particularly important, while in daily life scenes, scene change parameters are particularly important. Therefore, based on the scene type of the environment in which the wearable device is located, the adjustment coefficients corresponding to the user motion parameters, the event importance parameters, and the scene change parameters can be determined, so as to increase the proportion of the corresponding parameters in the priority as needed and improve the adjustment sensitivity of the shooting parameters.

[0039] Specifically, the adjustment coefficient of the user's motion parameters can be determined according to the user state of the user wearing the smart glasses or according to the scene to which the smart glasses belong. For example, when it is detected that the user is sedentary or sleeping, or the scene is a daily life scene, the adjustment coefficient of the user's motion parameters is reduced, and the adjustment coefficient of the scene change parameters and the event importance parameters is increased; when it is detected that the user is in high-intensity exercise such as running or playing ball, or in a medical monitoring scene, the adjustment coefficient of the user's motion parameters is increased, and the adjustment coefficient of the event importance parameters is reduced.

[0040] Remove the logarithm and use the pure multiplication formula as the priority calculation formula. The calculation formula of the priority P is:

[0041] in, Range of motion , Eye movement amplitude , is the scene change parameter 3 and is the importance parameter of the event Tuning parameters in the priority calculation, is the exponential parameter of the event importance parameter, which is used to amplify or suppress the event importance. When >1, the influence of the event importance parameter in the priority is amplified or suppressed exponentially; When <1, the effect of the event importance parameter in the priority is moderately smoothed; =1, the impact of the event importance parameter on the priority is linearly amplified.

[0042] It is understandable that the event importance parameter may be further calculated based on the user's biological parameters such as heart rate, blood pressure and body temperature, or the priority may be calculated based on the user's biological parameters such as heart rate, blood pressure and body temperature.

[0043] Through the above method, this embodiment determines the adjustment parameters corresponding to each parameter based on the importance of each parameter in different shooting scenes, thereby increasing the influence ratio of the corresponding parameter in the priority, and then adaptively adjusts the shooting parameters based on different parameters in different shooting scenes, thereby improving the adjustment accuracy of the shooting parameters.

[0044] Further, the obtaining of the event importance parameter of the wearable device includes: The event importance parameter is calculated based on a preset event scoring model, the user motion parameter and the scene change parameter.

[0045] The calculating of the event importance parameter based on the preset event scoring model, the user motion parameter and the scene change parameter includes: When the frame rate in the shooting parameters is lower than the minimum frame rate, based on a preset period, key frames are extracted from the captured images of the wearable device to obtain a key frame set, and based on the key frame set, the event scoring model is trained to obtain the event scoring model.

[0046] In this embodiment, the event scoring model is a multimodal model, which combines user motion parameters and scene change parameters to perform multi-dimensional scoring on the events in the image. First, images containing various subjects (such as faces, license plates, medical equipment, etc.) are screened out, and multi-dimensional analysis is performed based on the subject rarity (such as abnormal equipment, wild animals, etc.), the scene change amplitude of the fingertips of each frame image, abnormal behavior (such as sudden events such as falls, crowds, fires, etc.), the user's body movement amplitude, eye movement amplitude, heart rate change amplitude, and body temperature change amplitude, as well as the audio corresponding to the image in the video, to obtain event importance parameters. Among them, the more combined sub-events in the event, the higher the level of the event, that is, the higher the score, and the event with a score exceeding the threshold is a key event, such as motion intensity (ski jump), facial expression (smile) and audio excitement (cheers) to obtain a high-level key event, which can be marked as "best moment", and a high-level key event is obtained through audio (children's laughter) and user actions (chasing pictures), which can be marked as "family interaction moments".

[0047] Specific scoring rules include but are not limited to: the larger the movement, the higher the score of the corresponding event; the larger the face size of the target user in the image, the higher the score of the corresponding event; the higher the audio, the higher the score of the corresponding event, etc. Other dimensional parameters can also be combined, such as different shooting areas (such as home, hospital), different time periods (such as late at night, during meetings, etc.), to add points to events, and events can also be added based on user status (such as sudden increase in heart rate, abnormal body temperature, etc.); events can also be added based on specific people (such as the user himself, doctors or family members, etc.) or abnormal sounds (glass breaking, screaming, etc.).

[0048] In a specific embodiment, the event importance parameter may also be directly generated based on preset image semantics.

[0049] Exemplarily, the obtaining of the event importance parameter of the wearable device includes: The event importance parameter is generated based on the image semantics of the image taken by the wearable device.

[0050] In this embodiment, by extracting image keywords through OCR, such as text information in the picture or audio corresponding to the image (such as "danger" or "first aid"), or when the audio contains emotions such as anger or a cry for help, a high score can be triggered, that is, a high event importance score is generated.

[0051] In more embodiments, based on the event importance parameters of the two methods, the maximum score or the average score can be taken as the final event importance parameter.

[0052] It is understandable that the user can also customize the recording instructions of key events, such as clapping three times in front of the camera, to store key images with high resolution and high frame rate. When the recording instruction triggered by the user-defined operation is detected, the key image can be directly captured without priority calculation.

[0053] Before performing event scoring on the captured images through the event scoring model, key images stored in the smart glasses and their servers may be obtained to train the event scoring model to improve the scoring accuracy of the event scoring model.

[0054] In order to save the storage space of smart glasses, key images can be completely stored, and non-key images can be stored with the difference points from key images. Therefore, when it is detected that the shooting device has been running at the minimum frame rate, in order to prevent image storage abnormalities caused by too long intervals, key images are temporarily shot with high resolution and high frame rate when the duration reaches the preset time. Thus, the shooting device is controlled to extract key frames at fixed time intervals and temporarily store them at higher resolution, which can not only provide more training samples for the subsequent analysis of the multimodal model, but also prevent image storage abnormalities.

[0055] Through the above method, this embodiment uses image semantics or event scoring models to perform multimodal analysis on the event importance of the captured images from multiple dimensions such as the amplitude of change of user movement, the amplitude of change of user body state, the amplitude of change of shooting scene, sound and image semantics, thereby improving the accuracy of event importance scoring.

[0056] Reference Figure 3 , Figure 3 FIG. 4 is a flow chart of a third embodiment of a photographing method according to the present invention.

[0057] In this embodiment, the shooting parameter includes a first frame rate, and the step S40 includes: Step S41, calculating a second frame rate based on the target priority and the first frame rate; Step S42, obtaining a user mode of the wearable device, and determining a frame rate adjustment range of the shooting device in the user mode; Step S43: if the second frame rate belongs to the frame rate adjustment range, use the second frame rate as the adjusted frame rate, and shoot according to the second frame rate.

[0058] In this embodiment, user mode S5 includes energy saving mode, normal mode and sports mode. Different user modes S5 may correspond to different frame rate adjustment ranges, such as different user modes S5 may correspond to different maximum frame rates and the same minimum frame rate FR; different user modes S5 may also correspond to different maximum frame rates and different minimum frame rates. For example: The minimum frame rate of S5 for all user modes is set to one frame every 30 minutes, about 0.00056 FPS, the maximum frame rate of energy saving mode (S5=0) is set to 10FPS, the maximum frame rate of normal mode (S5=1) is set to 20 FPS, and the maximum frame rate of sports mode (S5=2) is set to 30 FPS.

[0059] In a specific embodiment, the maximum frame rate of each mode can also be dynamically adjusted based on the device status, such as the storage space of the device, the battery level of the device, etc., and the maximum frame rate of the sports mode, the maximum frame rate of the normal mode, and the maximum frame rate of the energy-saving mode are reduced in sequence.

[0060] Among them, the frame rate adjustment range is the maximum frame rate corresponding to the user mode To the minimum frame rate corresponding to each user mode The maximum frame rate corresponding to each user mode Do not exceed the maximum frame rate supported by the smart glasses , the minimum frame rate corresponding to each user mode Not less than the minimum frame rate supported by smart glasses .

[0061] Based on the target priority, the adjusted target frame rate, i.e., the second frame rate, is calculated, and shooting is performed according to the second frame rate. The second frame rate calculation formula is: ) is the current frame rate of the wearable device or the preset benchmark frame rate (such as the average frame rate), P is the target priority, is a fixed parameter.

[0062] Exemplarily, after acquiring the user mode of the wearable device and determining the frame rate adjustment range of the shooting device in the user mode, the method further includes: If the second frame rate does not belong to the frame rate adjustment range, when the second frame rate is greater than the maximum frame rate within the frame rate adjustment range, the maximum frame rate is used as the adjusted frame rate, and shooting is performed according to the maximum frame rate; When the second frame rate is less than a minimum frame rate within the frame rate adjustment range, the minimum frame rate is used as the adjusted frame rate, and shooting is performed according to the minimum frame rate.

[0063] In this embodiment, based on the second frame rate, the shooting frame rate of the smart glasses is controlled to dynamically change within the frame rate adjustment range corresponding to the user mode, that is, if the priority P is extremely low, FR is close to the minimum frame rate (one frame every 30 minutes); if P is extremely high, FR is close to the maximum frame rate in the current mode: When the second frame rate is not less than a minimum frame rate in the current user mode and not greater than a maximum frame rate in the current user mode, shooting based on the second frame rate; When the second frame rate is less than a minimum frame rate in the current user mode, shooting is performed based on the minimum frame rate in the current user mode; When the second frame rate is greater than the maximum frame rate in the current user mode, shooting is performed based on the maximum frame rate in the current user mode.

[0064] Furthermore, before obtaining the current user mode of the wearable device, the method further includes: Based on the user setting instruction, the device state parameter of the wearable device and / or the scene change parameter, the user mode of the wearable device is set or updated.

[0065] In this embodiment, the user mode can be adaptively adjusted based on the scene change corresponding to the shooting scene. If the scene change parameter is detected to be large, the current user mode is adjusted to the sports mode; if the scene change parameter is detected to be small, the current user mode is adjusted to the energy saving mode; if the scene change parameter is detected to reach the average value, the current user mode is adjusted to the normal mode.

[0066] After receiving the user's mode setting instruction, the current user mode is adjusted or set based on the user's voice instruction or the selection operation instruction in the device interface.

[0067] When detecting abnormal device status parameters of the smart glasses (such as insufficient storage space or insufficient power), the current user mode is automatically adjusted to energy-saving mode.

[0068] Further, the shooting parameters include a first resolution, and the step S40 includes: Calculating a second resolution based on the target priority and the first resolution; Obtaining a resolution adjustment range of the camera; If the second resolution belongs to the resolution adjustment range, the second resolution is used as the adjusted resolution, and shooting is performed according to the second resolution.

[0069] Wherein, adjusting the shooting parameters of the shooting device of the wearable device according to the target priority, and shooting according to the adjusted shooting parameters, includes: If the second resolution does not belong to the resolution adjustment range, then when the second resolution is greater than the maximum resolution within the resolution adjustment range, the maximum resolution is used as the adjusted resolution, and shooting is performed according to the maximum resolution; When the second resolution is smaller than a minimum resolution within the resolution adjustment range, the minimum resolution is used as the adjusted resolution, and shooting is performed according to the minimum resolution.

[0070] In this embodiment, the resolution is not affected by the user mode, and the resolution adjustment range is between the minimum resolution and the maximum resolution supported by the camera.

[0071] Based on the second resolution, the shooting resolution of the smart glasses is controlled to change dynamically within the resolution adjustment range (such as 360p to 4K). That is, when P is extremely high, the maximum resolution is selected for shooting; when P is extremely low, the minimum resolution is maintained for shooting to save power consumption: When the second resolution is not less than the minimum resolution and not greater than the maximum resolution, shooting is performed based on the second resolution; When the second resolution is smaller than the minimum resolution, shooting is performed based on the minimum resolution; When the second resolution is greater than the maximum resolution, photographing is performed based on the maximum resolution.

[0072] It is understandable that if a critical event is subsequently identified through the low-frequency loop (S4 increases significantly), the frame rate or resolution is continuously increased in subsequent recordings.

[0073] Through the above method, the logarithm is removed, the priority is calculated by pure multiplication, and the influence of key events on shooting parameters can be flexibly adjusted based on multiple dimensions. Further, by limiting the minimum frame rate, the waste of resources caused by continuous empty shooting of the camera device of the wearable device can be prevented, and the maximum frame rate can be further limited by the user mode to improve the compatibility of wearable devices with sports scenes and long-term recording scenes. In addition, the user mode is decoupled from the resolution to achieve independent adjustment of the resolution and precise control of various shooting parameters.

[0074] In combination with the above embodiments, an embodiment is further provided to illustrate a shooting method applied to a wearable device: 1. When the user is in a sedentary low-activity state and the event importance parameter corresponding to the wearable device is low (i.e., energy-saving mode + low event importance), if S1=01, S2=01, S3=0~5, S4=2 (low score, indicating a non-critical event), if =0.2, =0.3, =0.01, =0.5,α=1.2; then P≈(1+0.2×1)×(1+0.3×1)×(1+0.01×5)×(1+0.5×21.2)≈19.

[0075] If the P value is low, the frame rate is low (the minimum frame rate or slightly higher), the resolution is low (the minimum resolution or slightly higher), and the wearable device is in a low power consumption state; 2. When the user is in a normal state of walking on the street and the wearable device corresponds to a moderate event importance parameter (i.e. moderate exercise + moderate event importance in normal mode), S1=45, S2=35, S3=20~30, S4=40. The priority P is greatly increased to dozens or hundreds. The maximum limit of normal mode is 20 FPS, and the actual frame rate is close to 20 FPS. When the resolution rises to 720p or 1080p, the wearable device can capture clearer images. And when the event importance parameter S4 is increased, α brings an additional amplification effect to the event importance parameter.

[0076] 3. When the user is in a high-intensity running state in a sports venue and the wearable device corresponds to a high time importance parameter (i.e., sports mode + high event importance), S1=810, S2=59, S3=50~80, and the multimodal model analysis shows that S4=90, α=1.5\alpha=1.5, S4 more significantly improves the priority P. The frame rate can easily reach the upper limit of 30FPS in the sports mode, and the resolution can be increased to close to the maximum value of the device to ensure the complete capture of intense sports and important events. The S4 analysis strategy is frequently performed in this state to reduce the risk of missing key images (or key events) at the moment.

[0077] Among them, S1, S2, S3, and S4 are body movement amplitude data, eye movement amplitude data, the scene change parameter, and the event importance parameter, respectively. It is the exponential parameter of the event importance parameter, which is used to exponentially amplify or suppress the importance of the event.

[0078] Therefore, this application uses four major signals: body movement amplitude (S1), eye movement amplitude (S2), scene change (S3) and event importance (S4). Through a pure multiplicative priority formula, and introducing the exponent α in the S4 part, key events are amplified exponentially at high priority, thereby achieving high-fidelity capture. In addition, in any user mode, the minimum frame rate is not less than one frame every 30 minutes, the normal mode is up to 20 FPS, and the sports mode is up to 30 FPS. The resolution is flexibly selected according to the priority P and the device support range, thereby meeting the energy saving and clarity requirements in a variety of usage scenarios.

[0079] See also Figure 4 , Figure 4 1 is a schematic block diagram of a wearable device provided in an embodiment of the present application. The computer device may be a server.

[0080] See also Figure 4 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0081] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any shooting method.

[0082] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0083] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any shooting method.

[0084] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0085] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0086] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: Obtaining user motion parameters of the wearable device; Acquire a scene change parameter of the environment in which the wearable device is located, and acquire an event importance parameter of the wearable device; Calculating a target priority according to the user motion parameter, the scene change parameter and the event importance parameter; The shooting parameters of the shooting device of the wearable device are adjusted according to the target priority, and shooting is performed according to the adjusted shooting parameters.

[0087] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: Calculating a second frame rate based on the target priority and the first frame rate; Acquire a user mode of the wearable device, and determine a frame rate adjustment range of the shooting device in the user mode; If the second frame rate belongs to the frame rate adjustment range, the second frame rate is used as the adjusted frame rate, and shooting is performed according to the second frame rate.

[0088] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: If the second frame rate does not belong to the frame rate adjustment range, when the second frame rate is greater than the maximum frame rate within the frame rate adjustment range, the maximum frame rate is used as the adjusted frame rate, and shooting is performed according to the maximum frame rate; When the second frame rate is less than a minimum frame rate within the frame rate adjustment range, the minimum frame rate is used as the adjusted frame rate, and shooting is performed according to the minimum frame rate.

[0089] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: Based on the user setting instruction, the device state parameter of the wearable device and / or the scene change parameter, the user mode of the wearable device is set or updated.

[0090] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: Calculating a second resolution based on the target priority and the first resolution; Obtaining a resolution adjustment range of the camera; If the second resolution belongs to the resolution adjustment range, the second resolution is used as the adjusted resolution, and shooting is performed according to the second resolution.

[0091] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: If the second resolution does not belong to the resolution adjustment range, then when the second resolution is greater than the maximum resolution within the resolution adjustment range, the maximum resolution is used as the adjusted resolution, and shooting is performed according to the maximum resolution; When the second resolution is smaller than a minimum resolution within the resolution adjustment range, the minimum resolution is used as the adjusted resolution, and shooting is performed according to the minimum resolution.

[0092] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: The event importance parameter is calculated based on a preset event scoring model, the user motion parameter and the scene change parameter.

[0093] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: When the frame rate in the shooting parameters is lower than the minimum frame rate, based on a preset period, key frames are extracted from the captured images of the wearable device to obtain a key frame set, and based on the key frame set, the event scoring model is trained to obtain the event scoring model.

[0094] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: The event importance parameter is generated based on the image semantics of the image taken by the wearable device.

[0095] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: Acquire the scene type of the environment in which the wearable device is located, and determine the adjustment coefficients corresponding to the user motion parameter, the event importance parameter, and the scene change parameter respectively based on the scene type; The target priority is calculated based on the adjustment coefficients, the user motion parameter, the event importance parameter and the scene change parameter.

[0096] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: An index parameter of the event importance parameter is obtained, and the target priority is calculated based on the user motion parameter, the scene change parameter, the event importance parameter and the index parameter.

[0097] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: Obtaining parameter collection periods corresponding to the user motion parameter, the scene change parameter, and the event importance parameter respectively; Based on each parameter collection cycle, the user motion parameter, the scene change parameter and the event importance parameter are collected or updated respectively.

[0098] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: The user motion parameters include user body motion amplitude parameters and user eye motion amplitude parameters.

[0099] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a computer program. The computer program includes program instructions. The processor executes the program instructions to implement any one of the shooting methods provided in the embodiments of the present application.

[0100] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0101] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A shooting method, characterized in that: Applied to a wearable device, the shooting method includes: Obtaining user motion parameters of the wearable device; Acquire a scene change parameter of the environment in which the wearable device is located, and acquire an event importance parameter of the wearable device; Calculating a target priority according to the user motion parameter, the scene change parameter and the event importance parameter; The shooting parameters of the shooting device of the wearable device are adjusted according to the target priority, and shooting is performed according to the adjusted shooting parameters.

2. The shooting method according to claim 1, characterized in that: The shooting parameters include a first frame rate, and the step of adjusting the shooting parameters of the shooting device of the wearable device according to the target priority, and shooting according to the adjusted shooting parameters includes: Calculating a second frame rate based on the target priority and the first frame rate; Acquire a user mode of the wearable device, and determine a frame rate adjustment range of the shooting device in the user mode; If the second frame rate belongs to the frame rate adjustment range, the second frame rate is used as the adjusted frame rate, and shooting is performed according to the second frame rate.

3. The shooting method according to claim 2, characterized in that: After acquiring the user mode of the wearable device and determining the frame rate adjustment range of the shooting device in the user mode, the method further includes: If the second frame rate does not belong to the frame rate adjustment range, when the second frame rate is greater than the maximum frame rate within the frame rate adjustment range, the maximum frame rate is used as the adjusted frame rate, and shooting is performed according to the maximum frame rate; When the second frame rate is less than a minimum frame rate within the frame rate adjustment range, the minimum frame rate is used as the adjusted frame rate, and shooting is performed according to the minimum frame rate.

4. The shooting method according to claim 2, characterized in that: Before obtaining the current user mode of the wearable device, the method further includes: Based on the user setting instruction, the device state parameter of the wearable device and / or the scene change parameter, the user mode of the wearable device is set or updated.

5. The shooting method according to claim 1, characterized in that: The shooting parameters include a first resolution, and adjusting the shooting parameters of the shooting device of the wearable device according to the target priority, and shooting according to the adjusted shooting parameters, including: Calculating a second resolution based on the target priority and the first resolution; Obtaining a resolution adjustment range of the camera; If the second resolution belongs to the resolution adjustment range, the second resolution is used as the adjusted resolution, and shooting is performed according to the second resolution.

6. The shooting method according to claim 5, characterized in that: The step of adjusting the shooting parameters of the shooting device of the wearable device according to the target priority and shooting according to the adjusted shooting parameters includes: If the second resolution does not belong to the resolution adjustment range, then when the second resolution is greater than the maximum resolution within the resolution adjustment range, the maximum resolution is used as the adjusted resolution, and shooting is performed according to the maximum resolution; When the second resolution is smaller than a minimum resolution within the resolution adjustment range, the minimum resolution is used as the adjusted resolution, and shooting is performed according to the minimum resolution.

7. The shooting method according to claim 1, characterized in that: The step of obtaining the event importance parameter of the wearable device includes: The event importance parameter is calculated based on a preset event scoring model, the user motion parameter and the scene change parameter.

8. The shooting method according to claim 7, characterized in that: The calculating the event importance parameter based on the preset event scoring model, the user motion parameter and the scene change parameter includes: When the frame rate in the shooting parameters is lower than the minimum frame rate, based on a preset period, key frames are extracted from the captured images of the wearable device to obtain a key frame set, and based on the key frame set, the event scoring model is trained to obtain the event scoring model.

9. The shooting method according to claim 1, characterized in that: The step of obtaining the event importance parameter of the wearable device includes: The event importance parameter is generated based on the image semantics of the image taken by the wearable device.

10. The shooting method according to claim 1, characterized in that: The calculating the target priority according to the user motion parameter, the scene change parameter and the event importance parameter comprises: Acquire the scene type of the environment in which the wearable device is located, and determine the adjustment coefficients corresponding to the user motion parameter, the event importance parameter, and the scene change parameter respectively based on the scene type; The target priority is calculated based on the adjustment coefficients, the user motion parameter, the event importance parameter and the scene change parameter.

11. The shooting method according to claim 1, characterized in that: The calculating the target priority according to the user motion parameter, the scene change parameter and the event importance parameter comprises: An index parameter of the event importance parameter is obtained, and the target priority is calculated based on the user motion parameter, the scene change parameter, the event importance parameter and the index parameter.

12. The shooting method according to claim 1, characterized in that: The shooting method further includes: Obtaining parameter collection periods corresponding to the user motion parameter, the scene change parameter, and the event importance parameter respectively; Based on each parameter collection cycle, the user motion parameter, the scene change parameter and the event importance parameter are collected or updated respectively.

13. The shooting method according to any one of claims 1 to 12, characterized in that: The user motion parameters include user body motion amplitude parameters and user eye motion amplitude parameters.

14. A wearable device, characterized in that: The wearable device includes a processor, a memory, and a shooting program stored in the memory and executable by the processor, wherein when the shooting program is executed by the processor, the steps of the shooting method according to any one of claims 1 to 13 are implemented.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a shooting program, wherein when the shooting program is executed by a processor, the steps of the shooting method according to any one of claims 1 to 13 are implemented.

Images