AI glasses automatic shooting method and system based on voice control

Abstract

The invention provides an AI glasses automatic shooting method and system based on voice control, and relates to the technical field of intelligent wearable equipment, accurate interaction is realized through a multi-channel directional microphone array and an end-cloud collaborative voice recognition engine, the microphone array optimizes the pickup angle based on the wearing position characteristics of a user, and the user experience is improved. In combination with real-time voice activity detection, environmental noise is filtered out, it is ensured that a clear voice instruction can still be captured in a noisy environment, an end-cloud cooperation mode operates a lightweight model locally to guarantee the off-line response speed, a cloud large model is called when a network is available to improve the complex instruction analysis capability, and a composite statement containing parameter adjustment can be recognized; and an operation intention and parameters are automatically bound through a natural language processing engine, so that one-step execution of the instruction is realized.

Description

Technical Field

[0001] This invention relates to the field of smart wearable device technology, and in particular to an automated shooting method and system for AI glasses based on voice control. Background Technology

[0002] With the rapid development of smart wearable device technology, AI glasses, as an emerging terminal integrating visual interaction, environmental perception and information processing, have gradually become the focus of the consumer electronics field. Their lightweight design and first-person perspective shooting capabilities give them unique advantages in scenarios such as sports recording, outdoor adventure, and live streaming. Currently, mainstream AI glasses have basic shooting functions, but the operation methods are still mainly based on touch buttons or head motion sensing. In scenarios where users' hands are occupied, such as cycling or mountain climbing, it is difficult to quickly start shooting or adjust parameters, which restricts the practicality and scenario adaptability of the device.

[0003] Existing technologies for shooting devices with voice control functions have significant limitations. First, voice recognition only supports single commands such as "take a photo" or "record a video," and cannot parse complex commands that include parameter adjustments. Furthermore, it does not incorporate real-time data such as ambient light and motion status to optimize the recognition logic, resulting in a significant drop in recognition accuracy in noisy environments or when the user is moving. Second, the media processing and social sharing process lacks a closed-loop design. Users need to manually select photos and edit videos before being redirected to a third-party platform for uploading, which breaks the complete process from shooting to sharing. Moreover, it does not utilize GPS data to automatically generate scene tags, reducing the relevance and richness of content dissemination.

[0004] In summary, existing AI glasses shooting control methods suffer from limitations in operating scenarios, weak voice interaction capabilities, and non-closed-loop functional chains. To overcome these bottlenecks, it is urgent to build a fully automated shooting system based on voice control to meet users' needs for efficient creation and sharing in complex scenarios. Summary of the Invention

[0005] To address the technical problems existing in the prior art, the present invention provides the following technical solution:

[0006] On the one hand, a voice-controlled AI glasses automated shooting method is provided, which is implemented by an electronic device and includes:

[0007] S100: Collects voice command streams through a directional microphone array integrated into AI glasses; when a specific voice wake-up word is detected, it collects subsequent voice commands issued by the user.

[0008] S200. Input the voice command stream into the voice recognition engine, and combine it with the real-time acquired ambient light intensity data and motion sensor data to generate an executable command containing operation intent and parameter instructions. The intent includes shooting control instructions, media selection instructions, video editing instructions, and social sharing instructions. The parameter instructions include parameter adjustment instructions, content processing instructions, and target platform identifiers.

[0009] S300: Responding to the operation intent in the executable command, the AI ​​glasses are linked to control the shooting device to perform the corresponding operation;

[0010] S400: In response to the video editing instruction in the executable command, the intelligent video editor calls video key frames based on the time period located by S300 and performs cutting and splicing operations to generate the edited video file.

[0011] S500: In response to the social sharing instruction in the executable command, the preset social platform API interface is matched with the target platform identifier, and the target media file generated by S300 and S400 is automatically attached with an environmental tag generated by GPS data and uploaded.

[0012] Furthermore, the S100 continuously collects sound wave signals from the environment while the user is wearing the glasses by integrating a multi-channel directional microphone array into the temples and nose pads of the AI ​​glasses to form an initial voice stream.

[0013] The initial speech stream is processed to identify valid speech segments and filter out background noise;

[0014] Within the detected valid speech segments, a local preset wake word recognition model is run to identify specific preset speech wake words;

[0015] When the specific voice wake-up word is successfully recognized, the command acquisition state of the AI ​​glasses is immediately activated, and the voice commands issued by the user within the subsequent time window are continuously acquired to form the voice command stream.

[0016] During the voice command acquisition process, the audio timestamps corresponding to the microphone array are recorded synchronously and associated with the voice command stream.

[0017] Furthermore, the S200 synchronously acquires voice command streams, ambient light intensity data, motion sensor data, and GPS positioning data through the sensors integrated into the AI ​​glasses;

[0018] Input the voice command stream into the cloud-based collaborative speech recognition engine, run the lightweight speech recognition engine locally, and generate preliminary text commands;

[0019] When the network is available, the audio stream is uploaded to the cloud in segments, the speech recognition engine is run to generate optimized text, and the optimized text is merged with the local results to generate text commands;

[0020] The generated text commands are input into the natural language processing engine, and the output is a structured executable command containing: operation intent and parameter instructions.

[0021] Furthermore, the natural language processing engine parses the operation intent through entity recognition and intent classification, generates executable commands, and binds the operation intent and parameter instructions in the executable commands to jointly execute the user commands. The rules for generating the operation intent are as follows:

[0022] When the text command contains the configurable verb {record video}, output the shooting control command;

[0023] When the text command contains the configurable verb {select keep delete}, output the media selection instruction;

[0024] When the text command contains the configurable verb {editing acceleration}, output video editing instructions;

[0025] When the text command contains the configurable verb {share, publish, upload}, output the social sharing command;

[0026] The parameter command binding logic is as follows: shooting control commands are bound to parameter adjustment commands, media selection commands and video editing commands are bound to content processing commands, and social sharing commands are bound to the target platform identifier.

[0027] Furthermore, the S300 responds to the operational intent in the executable command. When the intent is a shooting control command, it parses the parameter adjustment command and extracts the specific parameter values ​​of brightness, contrast, focal length, and exposure time.

[0028] The parameter configuration interface of the vision processing engine is called to convert the extracted parameter values ​​into electrical signal instructions that can be recognized by the image sensor. The brightness parameter corresponds to adjusting the ISO sensitivity of the image sensor, the contrast parameter corresponds to adjusting the gamma curve of the image signal processor, and the focal length parameter achieves optical zoom by controlling the stepper motor of the camera.

[0029] When taking a picture, the visual processing engine triggers the image sensor to perform a single-frame exposure. After the exposure is complete, the image data is transmitted to a temporary buffer. After autofocus and white balance calibration, a raw photo file in JPEG format is generated.

[0030] During the recording operation, the visual processing engine controls the image sensor to continuously acquire frames and compress them in real time to generate the original video file, while recording the timestamp of each frame and the corresponding sensor parameters.

[0031] After the original media file is generated, metadata tags such as shooting time, geographical location, and device model are automatically added and stored in the media storage and tagging unit.

[0032] Furthermore, in S300, when the intent is a media selection instruction and it is directed at a photo, the intelligent photo selection analyzer reads a set of candidate photos from the media storage and tagging unit and extracts the image quality features of each photo, including the sharpness value, color saturation, and face detection results.

[0033] Based on the filtering conditions in the content processing instructions, candidate photos are filtered and marked as initial targets. The scene type in the photos is identified by the sensor and matched with the scene description in the user instructions to further filter out the final target photos.

[0034] Selected target photos are tagged with "Keep" and stored in the media storage and tagging unit. Unselected photos are tagged with "Pending Processing" and await further instructions from the user.

[0035] Furthermore, in S400, the intelligent video editor retrieves the time period information located in S300 from the video segment index library to determine the range of the video segment to be edited;

[0036] The video keyframes are used to identify specific time points such as transition points and motion peaks within the clip, which serve as reference nodes for editing and splicing.

[0037] The original video is segmented and reassembled according to the editing requirements in the content processing instructions;

[0038] After editing, a multi-track video file containing the main video track, audio track, and subtitle track is generated, allowing users to further adjust track parameters via voice commands.

[0039] On the other hand, an automated shooting system for AI glasses based on voice control is provided. This system is applied to the automated shooting method for AI glasses based on voice control. The system includes: a voice acquisition module, a command parsing module, a shooting control module, a media processing module, and a social sharing module.

[0040] The voice acquisition module is integrated into the temples and nose pads of the AI ​​glasses. It includes a multi-channel directional microphone array and a real-time voice activity detection unit. It is used to continuously acquire ambient sound wave signals to form an initial voice stream, detect specific voice wake words through a local preset wake word recognition model, activate the command acquisition state and generate a voice command stream, and synchronously record audio timestamps.

[0041] The instruction parsing module integrates a speech recognition engine and a natural language processing engine. After receiving the voice command stream, it combines real-time data obtained by the ambient light sensor, motion sensor and GPS module to generate text commands through edge-cloud collaborative recognition. After entity recognition and intent classification, it parses out the operation intent and parameter instructions, forms a structured executable command and completes instruction binding.

[0042] The shooting control module includes a visual processing engine and an image sensor. When responding to shooting control commands, it parses parameter adjustment commands and converts them into electrical signals, controls the image sensor to perform photo and video recording operations, processes the image data, generates raw media files with metadata tags, and stores them in the media storage and tagging unit.

[0043] The media processing module consists of an intelligent video editor and a media storage and tagging unit. When responding to a media selection instruction, it extracts photo quality features and filters target photos, locates the target time period of the video, and when responding to a video editing instruction, it performs video segmentation and recombination based on the location information to generate a multi-track edited video file.

[0044] The social sharing module includes a platform interface adaptation unit and a tag generation unit. When responding to a social sharing command, it matches the target platform identifier with a preset API interface, performs format adaptation processing on the target media file, generates an environment tag through GPS data, and completes the media file upload after encryption and authentication.

[0045] On the other hand, an electronic device is provided, the electronic device comprising: a processor and a memory;

[0046] The memory stores computer-readable instructions, which, when executed by the processor, implement any of the methods described above for automated shooting with AI glasses based on voice control.

[0047] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods of automated shooting for AI glasses based on voice control.

[0048] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0049] I. This invention achieves accurate interaction through a multi-channel directional microphone array and an edge-cloud collaborative speech recognition engine. The microphone array optimizes the sound pickup angle based on the user's wearing position characteristics and filters out environmental noise by combining real-time voice activity detection, ensuring that clear voice commands can still be captured in noisy environments. The edge-cloud collaborative mode runs a lightweight model locally to ensure offline response speed, and calls a large model in the cloud when the network is available to improve the ability to parse complex commands. It can recognize compound sentences containing parameter adjustments and automatically bind operation intentions and parameters through a natural language processing engine to achieve one-step execution of commands.

[0050] Second, the media processing module and the shooting control module of the present invention form a linkage mechanism, which automatically completes content optimization based on semantic analysis results. When a user instruction is received, the intelligent photo selection analyzer extracts features such as image clarity and dynamic blur, and combines them with the acceleration data recorded by the motion sensor to accurately identify the scene that meets the conditions. In the video editing stage, the motion peak point is called to automatically match the editing instruction and generate a segment with a suitable rhythm, thereby realizing intelligent and precise content processing. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of an automated shooting method for AI glasses based on voice control provided in an embodiment of the present invention;

[0053] Figure 2 This is a flowchart of the automated shooting method for AI glasses based on voice control provided in an embodiment of the present invention;

[0054] Figure 3 This is a block diagram of an AI glasses automated shooting system based on voice control provided in an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0056] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0057] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0058] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0059] In this embodiment of the invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0060] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0061] Example 1

[0062] This invention provides an automated shooting method for AI glasses based on voice control. This method can be implemented by an electronic device, which can be a terminal or a server. Figure 2 The flowchart shown below illustrates the steps of an automated shooting method for AI glasses based on voice control. The working principle of this method is as follows:

[0063] First, the system enters the voice command stream acquisition stage (S100). In this stage, the AI ​​glasses use an integrated directional microphone array to acquire, process, and recognize the voice signal, laying the foundation for subsequent command parsing. The specific working principle is as follows:

[0064] The AI ​​glasses integrate a multi-channel directional microphone array in the temples and nose pads, which can more accurately capture the sound wave signals emitted by the user when the user is wearing them normally. When the user wears the AI ​​glasses, the microphone array will continue to work, constantly collecting sound waves in the surrounding environment. These raw sound wave signals will form the initial speech stream.

[0065] The initial audio stream contains not only the voice that the user may make, but also a lot of background noise, such as the sound of wind, conversations of people around, and the roar of vehicles. To extract effective speech information, the audio stream is segmented into frames (20ms frame length, 10ms frame shift). The short-time energy (calculated by the sum of squares of the signal within the frame) and zero-crossing rate (number of positive and negative signal alternations within the frame) of each frame are analyzed. When the real-time speech activity detection unit starts, it first collects 500ms of ambient noise without speech and performs effective speech segment identification. The energy and zero-crossing rate of each frame are compared with the noise, and a condition judgment is made: when the short-time energy exceeds 3 times the average noise energy and the zero-crossing rate is less than 60% of the noise zero-crossing rate, it is judged as effective speech. When the condition is met for 10 consecutive frames (100ms), it is marked as the start point of the effective speech segment. When the condition is not met for 20 consecutive frames (200ms), it is marked as the end point of the effective speech segment. By analyzing the characteristics of the speech signal (such as short-time energy, zero-crossing rate, etc.), the effective speech segment, that is, the part of the speech actually spoken by the user, can be identified, and the background noise is filtered out, thus obtaining a relatively pure effective speech segment.

[0066] Within the obtained valid speech segments, a locally pre-built wake word recognition model is run. The wake word model construction process is as follows: the locally pre-built wake word recognition model adopts a deep neural network (DNN) architecture, with 13-dimensional frequency coefficients as input, 3 hidden layers (256 neurons per layer), and a binary classification layer ("wake word" or "non-wake word"). The model training data includes wake word samples recorded by the user in different environments (quiet, noisy, and moving), as well as non-wake word speech samples. The valid speech segments are divided into 100ms sliding windows, and the window data is input into the wake word model. If the probability of the wake word output by the model in each frame within the window is greater than 0.95 for 5 consecutive frames, activation is triggered, and the command acquisition state is activated. After activation, a 5-second time window is opened to continuously acquire the user's subsequent voice and form a voice command stream. When a specific voice wake word is successfully recognized, the AI ​​glasses will immediately switch from standby state to command acquisition state. At this time, a time window is opened to continuously acquire the voice commands issued by the user. These voice commands together constitute the voice command stream. This design ensures that the user can issue subsequent operation commands in a timely manner after saying the wake word.

[0067] During the acquisition of voice commands, the audio timestamps corresponding to each microphone are recorded synchronously. The recording of timestamps is to enable the accurate correlation of voice information with other sensor data (such as motion sensor data, ambient light data, etc.) in the time dimension when processing the voice command stream, so as to ensure the synchronization of various data and provide accurate time reference for subsequent command parsing and operation execution.

[0068] Then, the executable command generation stage (S200) begins. This stage processes the acquired voice command stream and combines it with various sensor data to generate structured executable commands. The specific working principle is as follows:

[0069] AI glasses integrate multiple sensors, including a microphone array, ambient light sensor, motion sensor (such as accelerometer and gyroscope), and GPS positioning module. While processing the voice command stream, these sensors work synchronously to acquire ambient light intensity data (reflecting the lighting conditions of the surrounding environment), motion sensor data (recording the movement state of the AI ​​glasses, such as whether they are moving and their speed), and GPS positioning data (determining the user's current geographical location). Based on the start timestamp of the voice command stream, the sampling data of each sensor within ±5ms is searched, and the matched ambient light, motion, and GPS data are packaged into "contextual information" and appended to the metadata field of the voice command stream to form a multi-dimensional dataset.

[0070] The voice command stream is processed by an input-side cloud-based collaborative speech recognition engine. This engine employs a local and cloud-based collaborative working mode: First, a lightweight speech recognition engine runs locally. This engine connects the voice signal with the text command, using a collaborative architecture of "local lightweight model + cloud-based large model" to balance response speed and recognition accuracy. It utilizes a deep residual network ResNet and a bidirectional long short-term memory network BiLSTM. The input layer receives 13 frequency features, the ResNet layer (3 residual blocks) extracts spectral features, the BiLSTM layer (2 layers, 128 neurons per layer) captures temporal dependencies, and the output layer achieves unaligned speech recognition. The system collects relevant speech samples from the shooting scene, covering different genders, ages, and accents. Twenty types of environmental noise are mixed in to generate noisy samples, enhancing anti-interference capabilities. Parameters are compressed to adapt to the local storage and computing resources of the AI ​​glasses. When a voice command stream is input, it is first segmented into frames (20ms / frame) and spectral features are extracted. These are then input into the local model. The model filters noise interference through a ResNet layer and analyzes the temporal features of the speech through a BiLSTM layer. Finally, CTC decoding generates preliminary text. For example, if a user says "turn the brightness up by 30%", the local model outputs the probability of the text "turn the brightness up by 30%" (if the confidence level is ≥0.85, it is directly adopted; otherwise, it is marked as "to be optimized"). The advantage of the local lightweight engine is that it can guarantee basic speech recognition functions even without a network, ensuring the system's availability.

[0071] When network availability is detected, the audio stream is segmented and uploaded to the cloud. The speech recognition engine running in the cloud has more powerful processing capabilities and a richer corpus, enabling more accurate recognition of audio segments. It adopts a Transformer architecture (12-layer encoder, 6-layer decoder), with 80-dimensional spectral features as input. By capturing long-distance speech dependencies, it supports contextual semantic understanding. It is trained based on general speech data and shooting domain data, and optimizes dictionary terminology for shooting scenarios to improve recognition accuracy. When network availability is available, the local audio command stream is segmented (10 seconds per segment) and uploaded to the cloud. The cloud model re-extracts features from the audio, decodes the speech sequence through the Transformer encoder, and the decoder combines with the language model to generate optimized text. For example, if the local model mistakenly recognizes "exposure duration 1 second" as "exposure time 1 second", the cloud model corrects it to the correct text through contextual semantics. Finally, the cloud returns the optimized text to the local machine to generate optimized text commands. The optimized text commands generated by the cloud are then fused with the preliminary text commands generated locally, combining the advantages of both to finally generate accurate text commands.

[0072] The generated text commands are input into a natural language processing engine, which parses the user's operational intent through entity recognition and intent classification to generate structured executable commands. The construction process is as follows: A BiLSTM-CRF architecture is used. The input is the word vector of the text command. A BiLSTM layer (2 layers, 256 neurons each) extracts contextual features, and the CRF layer outputs entity labels. The training data consists of labeled samples. Local text features are extracted, and after global max pooling, they are input into a softmax layer, outputting four types of intents (shooting control, media selection, video editing, and social sharing). The training data consists of text commands with intent labels and predefined parameter binding rules. For example, if the intent is "shooting control," it binds parameters such as "brightness" and "focal length" from the entity recognition results; if the intent is "social sharing," it binds the "target platform" entity. The rule engine binds the intent to the entity and generates structured executable commands. The executable commands contain the operational intent and parameter instructions, and these two are bound together to execute the user's command.

[0073] The generation of operation intentions follows these rules: when the text command contains configurable verbs {shoot, capture, record}, ​​the natural language processing engine will output shooting control instructions; when it contains {select, keep, delete}, it will output media selection instructions; when it contains {cut, composite, speed up}, it will output video editing instructions; and when it contains {share, publish, upload}, it will output social sharing instructions.

[0074] The binding logic for parameter commands is as follows: shooting control commands are bound to parameter adjustment commands, which involve parameters such as brightness, contrast, focal length, and exposure time; media selection commands and video editing commands are bound to content processing commands, which include requirements for filtering, segmenting, and reorganizing media content; and social sharing commands are bound to target platform identifiers, which specify the social platform on which the user wants to share the content.

[0075] Next, the camera device linkage operation stage (S300) begins. In this stage, based on the operation intent in the generated executable command, the AI ​​glasses are linked to control the camera device to perform the corresponding operation. The specific working principle is as follows:

[0076] When the operation intent in the executable command is a shooting control instruction, the bound parameter adjustment instruction is first parsed, and the specific parameter values ​​of brightness, contrast, focal length, and exposure time are extracted from the instruction.

[0077] The parameter configuration interface of the visual processing engine is called to convert the extracted parameter values ​​into electrical signal commands that the image sensor can recognize. Among them, the brightness parameter corresponds to adjusting the ISO sensitivity of the image sensor. The higher the ISO sensitivity, the stronger the image sensor is to light, and the brighter the picture can be captured in darker environments. The contrast parameter corresponds to adjusting the gamma curve of the image signal processor. The adjustment of the gamma curve affects the contrast between the bright and dark parts of the picture, making the picture more in line with the user's visual needs. The focal length parameter realizes optical zoom by controlling the stepper motor of the camera. The stepper motor can precisely drive the lens to move, thereby changing the focal length and realizing clear shooting of distant or near objects.

[0078] During the photo-taking process, the visual processing engine triggers the image sensor (12 megapixels) to perform a single-frame exposure. The exposure time is automatically adjusted according to the ambient light (range 1 / 10000s-1s). The image data is first stored in a temporary buffer, and then autofocus is performed to focus on the main subject of the image (such as a face or moving object) with a focus accuracy of ≤0.1mm. The type of light source (sunlight, fluorescent light, etc.) is determined by the ambient light sensor data, and the RGB channel gain is adjusted to ensure accurate white reproduction. Finally, a JPEG format photo (85% compression quality) with a resolution of 4000×3000 is generated. The image sensor continuously captures images at a frame rate of 30fps. Through real-time compression, an MP4 format video is generated. Each frame is appended with a timestamp, ISO, focal length, and other sensor parameters, and stored in 64GB of internal storage (supporting UHS-I high-speed read and write). Through exposure and autofocus processing, the sharpness of the subject is ensured. At the same time, white balance calibration is performed to ensure that the white in the image is accurately presented under different lighting conditions, avoiding color cast. After these processes, the original JPEG format photo file is generated.

[0079] During recording, the visual processing engine controls the image sensor to capture consecutive frames. These consecutive frame data are compressed in real time to reduce storage usage and facilitate subsequent processing, ultimately generating the original video file. During the recording process, the timestamp of each frame and the corresponding sensor parameters are recorded simultaneously.

[0080] After the original media files (including photos and videos) are generated, metadata tags are automatically added to them. These tags include information such as shooting time, geographical location (based on GPS positioning data), and device model. Then, the media files with metadata tags are stored in the media storage and tagging unit so that subsequent media selection, editing and other operations can be quickly located and retrieved.

[0081] When the operation intent is a media selection command targeting photos, the intelligent photo selection analyzer reads a set of candidate photos from the media storage and tagging unit. The intelligent photo selection analysis is based on video keyframes and user commands, achieving automated cutting and splicing. It uses a lightweight CNN to extract video frame features, calculates the feature cosine similarity of adjacent frames, and marks transition points when the similarity is <0.6. It also marks motion peaks by combining motion sensor data (acceleration >2g). The training data consists of videos of sports and outdoor scenes, with transitions and peaks manually labeled. Predefined editing logic is used, such as: "Accelerate" command: increases the video frame rate from 30fps to 60fps and removes redundant frames; "Splice" command: adds a 0.5-second fade-in / fade-out effect at transition points to ensure smoothness; "Multi-track synthesis": automatically matches the main video track and the ambient sound track (e.g., retaining wind sounds and conversations during movement). Input is the original video file and editing commands (e.g., "from the 10th second to..."). At 20 seconds, the video is spliced ​​1.5 times faster, and keyframes are stitched together. The engine traverses the video frames and uses a keyframe recognition model to locate transition points (such as when the scene changes from "mountain road" to "mountain top") and motion peaks (such as when the user jumps) within 10-20 seconds. Cutting: Deletes segments before 10 seconds and after 20 seconds; Speeding up: Compresses the 10-second segment to 6.7 seconds (1.5x speed); Stitching: Stitches the segments in chronological order with transition points as boundaries, adding transition effects; Multi-track generation: Automatically separates the audio track of the original video to generate "video main track + ambient sound track + subtitle track" (subtitles are generated by speech recognition), supporting subsequent voice adjustments (such as "increase the volume by 20%). The analyzer extracts image quality features from each photo, including sharpness values ​​(determined by analyzing the edge sharpness of the image, etc.), color saturation (reflecting the vibrancy of the colors in the photo), and face detection results (determining whether there is a face in the photo and the clarity of the face, etc.).

[0082] Based on the filtering criteria in the content processing instructions, candidate photos are initially screened and marked as preliminary targets. Simultaneously, the system uses sensors to identify the scene type in the photos and matches it with the scene description in the user instructions to further filter out the final target photos.

[0083] For the selected target photos, a "keep" tag is added and they are stored in the media storage and tagging unit, while unselected photos are tagged as "pending processing" and await further instructions from the user.

[0084] Next, the video editing process (S400) begins. This stage responds to video editing instructions in the executable commands, edits the video, and generates a edited video file. The specific working principle is as follows:

[0085] The intelligent video editor retrieves the time segment information located by the S300 stage from the video segment index library. This time segment information is determined based on the user's voice commands, thereby clarifying the range of video segments to be edited.

[0086] The video keyframe recognition function is invoked to identify special time points within the clip. These special time points include transition points (the moments when the scene changes in the video) and motion peaks (the moments when the movement of the image in the video is the most intense). These special time points will serve as reference nodes for editing and splicing, making the joints of the edited video more natural and smooth.

[0087] Based on the editing requirements in the content processing instructions, the intelligent video editor will segment and reassemble the original video. For example, for the requirement of acceleration, the frame rate of the video will be adjusted to achieve accelerated playback. For the requirement of deleting duplicate segments, the content similarity of different segments will be compared, the duplicate parts with high similarity will be deleted, and the remaining parts will be reassembled in chronological order.

[0088] After editing, a multi-track video file is generated, containing a main video track, an audio track, and a subtitle track. The main video track contains the edited video footage; the audio track retains the original video audio or adds new audio based on user instructions; and the subtitle track contains subtitles generated based on speech recognition or user-specified subtitle content. Additionally, users can further adjust track parameters via voice commands.

[0089] Finally, the social sharing upload stage (S500) begins. This stage responds to social sharing commands and uploads the processed media file to the target social platform. The specific working principle is as follows:

[0090] Upon receiving a social sharing instruction, the system matches the target platform identifier bound in the instruction with a pre-set social platform API interface library to find the corresponding API interface for the target platform, thereby enabling effective communication with the platform.

[0091] The original media files generated in the S300 stage and the edited video files generated in the S400 stage are processed for format adaptation. According to the specific requirements of the target platform, the media files are converted and compressed to ensure that the files can meet the platform's upload standards.

[0092] At the same time, environmental tags are generated using GPS positioning data. These environmental tags contain information about the shooting location of the media files and the environmental characteristics at the time of shooting.

[0093] The processed media files, which are automatically tagged with environmental information, are uploaded to the target social media platform via the matched social media platform API. During the upload process, encryption and authentication are performed to ensure the security of data transmission and prevent the media files from being tampered with or leaked during transmission. After the upload is completed, the sharing result is fed back to the user.

[0094] In summary, this embodiment details the complete process of an automated shooting method for AI glasses based on voice control. From S100, voice command stream acquisition ensures accurate capture of user instructions through a directional microphone array and wake-word recognition; to S200, combining multi-sensor data and edge-cloud collaborative voice recognition to generate executable commands, achieving accurate interpretation of user intent; then, in S300, the shooting equipment is linked to perform shooting and media selection operations, ensuring media content quality and accurate filtering; next, in S400, intelligent video editing generates multi-track videos that meet user needs; finally, in S500, social sharing of media files is completed, forming a complete system from shooting to sharing. The entire process fully utilizes AI technology, sensor technology, and network communication technology, achieving efficient, convenient, and intelligent shooting and sharing in complex scenarios, greatly improving the user experience and providing strong technical support for the application of smart wearable devices in the field of shooting.

[0095] Example 2

[0096] This embodiment provides the specific steps of an AI glasses automated shooting method based on voice control, such as... Figure 1 As shown, the specific steps of this method are as follows:

[0097] I. Voice command stream acquisition

[0098] A multi-channel directional microphone array integrated into the temples and nose pads of the AI ​​glasses continuously collects ambient sound signals while the user wears them, forming an initial voice stream.

[0099] The initial speech stream is input into the real-time speech activity detection module to identify valid speech segments and filter out background noise.

[0100] In the valid voice segment, run the locally pre - installed wake - word recognition model to recognize the preset specific voice wake - word.

[0101] If the wake - word is successfully recognized, immediately activate the command collection state of the AI glasses and continuously collect the user's voice commands within the subsequent time window to form a voice command stream.

[0102] When collecting voice commands, synchronously record the audio timestamps of the microphone array and associate them with the voice command stream.

[0103] II. Generation of Executable Commands

[0104] The sensors integrated in the AI glasses synchronously obtain the voice command stream, ambient light intensity data, motion sensor data, and GPS positioning data.

[0105] Input the voice command stream into the cloud - edge collaborative speech recognition engine:

[0106] Run the lightweight speech recognition engine locally to generate preliminary text commands.

[0107] When the network is available, segmentally upload the audio stream to the cloud. The cloud speech recognition engine generates optimized text and merges it with the local result into the final text command.

[0108] Input the text command into the natural language processing engine to parse the operation intention through entity recognition and intent classification:

[0109] If the text contains {shoot, record, video}, output a shooting control command.

[0110] If the text contains {select, retain, delete}, output a media selection command.

[0111] If the text contains {cut, synthesize, accelerate}, output a video editing command.

[0112] If the text contains {share, publish, upload}, output a social sharing command.

[0113] Bind the operation intention and parameter commands:

[0114] Bind the shooting control command with the parameter adjustment command.

[0115] Bind the media selection command and the video editing command with the content processing command.

[0116] Bind the social sharing command with the target platform identifier to form a structured executable command.

[0117] III. Execute Shooting and Related Operations

[0118] Respond to the shooting control command:

[0119] Parse the parameter adjustment commands to extract the specific parameter values ​​for brightness, contrast, focal length, and exposure time.

[0120] The visual processing engine parameter configuration interface is called to convert the parameter values ​​into electrical signals that can be recognized by the image sensor (brightness corresponds to ISO sensitivity, contrast corresponds to gamma curve, and focal length is achieved by stepper motor to achieve optical zoom).

[0121] Take a picture: Trigger a single-frame exposure of the image sensor, transmit the image data to a temporary buffer, and generate a raw JPEG image after autofocus and white balance calibration.

[0122] Recording: Control the image sensor to continuously acquire frames and compress them in real time to generate raw video files, recording the timestamp of each frame and sensor parameters.

[0123] The original media files are automatically tagged with metadata such as shooting time, geographical location, and device model, and stored in the media storage and tagging unit.

[0124] Responding to media selection instructions (for photos):

[0125] The intelligent photo selection analyzer reads a set of candidate photos from the media storage and tagging unit and extracts image quality features such as sharpness, color saturation, and face detection results.

[0126] Based on the filtering criteria of the content processing instructions, filter and mark the initial target photos.

[0127] The sensor identifies the scene type of the photo and matches it with the scene description in the user's instructions to further filter out the final target photo.

[0128] Add a "Keep" tag to the target photo and save it; add a "Pending Processing" tag to unselected photos.

[0129] IV. Video Editing Operations

[0130] The intelligent video editor retrieves the time period information located by the S300 from the video clip index library to determine the range of video clips to be edited.

[0131] The video keyframes are used to identify special time points such as transition points and motion peaks within the clip, which are then used as reference nodes for editing and splicing.

[0132] The original video is segmented and reassembled according to the editing requirements of the content processing instructions.

[0133] After editing, a multi-track video file containing the main video track, audio track, and subtitle track is generated, allowing users to further adjust track parameters via voice commands.

[0134] V. Social Sharing Operation

[0135] In response to social sharing commands, the target platform identifier is matched with the pre-defined social platform API interface.

[0136] The system automatically attaches environmental tags generated from GPS data to the original media files generated by the S300 and the edited video files generated by the S400.

[0137] The processed media files are uploaded to the target social media platform via the matching API interface.

[0138] Figure 3 This is a block diagram of a voice-controlled AI glasses automated shooting system, which is used for automated shooting methods of AI glasses based on voice control. (Refer to...) Figure 3 The system includes a voice acquisition module, a command parsing module, a shooting control module, a media processing module, and a social sharing module, among which:

[0139] The voice acquisition module is integrated into the temples and nose pads of the AI ​​glasses. It includes a multi-channel directional microphone array and a real-time voice activity detection unit. It is used to continuously acquire ambient sound wave signals to form an initial voice stream, detect specific voice wake words through a local pre-set wake word recognition model, activate the command acquisition state and generate a voice command stream, and simultaneously record audio timestamps.

[0140] The command parsing module integrates a speech recognition engine and a natural language processing engine. After receiving the voice command stream, it combines real-time data obtained from the ambient light sensor, motion sensor and GPS module, and generates text commands through edge-cloud collaborative recognition. After entity recognition and intent classification, it parses out the operation intent and parameter instructions, forms a structured executable command and completes the command binding.

[0141] The shooting control module includes a visual processing engine and an image sensor. When responding to shooting control commands, it parses parameter adjustment commands and converts them into electrical signals, controls the image sensor to perform photo and video recording operations, processes the image data, generates raw media files with metadata tags, and stores them in the media storage and tagging unit.

[0142] The media processing module consists of an intelligent video editor and a media storage and tagging unit. When responding to media selection instructions, it extracts photo quality features and filters target photos, locates the target time segment of the video, and when responding to video editing instructions, it performs video segmentation and reassembly based on the location information to generate multi-track edited video files.

[0143] The social sharing module includes a platform interface adaptation unit and a tag generation unit. When responding to a social sharing command, it matches the target platform identifier with the preset API interface, performs format adaptation processing on the target media file, generates an environment tag through GPS data, and completes the media file upload after encryption and authentication.

[0144] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 4 As shown, the electronic device may include the above-mentioned Figure 3 The illustrated AI glasses automated shooting system is based on voice control. Optionally, the electronic device 410 may include a first processor 2001.

[0145] Optionally, the electronic device 410 may also include a memory 2002 and a transceiver 2003.

[0146] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0147] The following is combined Figure 4 A detailed description of each component of electronic device 410 is provided below:

[0148] The first processor 2001 is the control center of the electronic device 410. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0149] Optionally, the first processor 2001 can perform various functions of the electronic device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0150] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0151] In a specific implementation, as one example, the electronic device 410 may also include multiple processors, for example... Figure 4The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0152] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0153] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be accessed through the interface circuit of the electronic device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0154] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0155] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0156] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected via the interface circuit of the electronic device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0157] It should be noted that, Figure 4 The structure of the electronic device 410 shown does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0158] Furthermore, the technical effects of the electronic device 410 can be referenced from the technical effects of the AI ​​glasses automated shooting method based on voice control described in the above method embodiments, and will not be repeated here.

[0159] It should be understood that the first processor 2001 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0160] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0161] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0162] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An automated shooting method for AI glasses based on voice control, characterized in that, The method includes: S100: Collects voice command streams through a directional microphone array integrated into AI glasses; when a specific voice wake-up word is detected, it collects subsequent voice commands issued by the user. S200. Input the voice command stream into the voice recognition engine, and combine it with the real-time acquired ambient light intensity data and motion sensor data to generate an executable command containing operation intent and parameter instructions. The intent includes shooting control instructions, media selection instructions, video editing instructions, and social sharing instructions. The parameter instructions include parameter adjustment instructions, content processing instructions, and target platform identifiers. S300: Responding to the operation intent in the executable command, the AI ​​glasses are linked to control the shooting device to perform the corresponding operation; S400: In response to the video editing instruction in the executable command, the intelligent video editor calls video key frames based on the time period located by S300 and performs cutting and splicing operations to generate the edited video file. S500: In response to the social sharing instruction in the executable command, the preset social platform API interface is matched with the target platform identifier, and the target media file generated by S300 and S400 is automatically attached with an environmental tag generated by GPS data and uploaded.

2. The automated shooting method for AI glasses based on voice control according to claim 1, characterized in that, The S100 uses a multi-channel directional microphone array integrated into the temples and nose pads of the AI ​​glasses to continuously collect sound wave signals from the environment while the user is wearing the glasses, forming an initial voice stream. The initial speech stream input is processed by the real-time speech activity detection unit to identify valid speech segments and filter out background noise; Within the detected valid speech segments, a local preset wake word recognition model is run to identify specific preset speech wake words; When the specific voice wake-up word is successfully recognized, the command acquisition state of the AI ​​glasses is immediately activated, and the voice commands issued by the user within the subsequent time window are continuously acquired to form the voice command stream. During the voice command acquisition process, the audio timestamps corresponding to the microphone array are recorded synchronously and associated with the voice command stream.

3. The automated shooting method for AI glasses based on voice control according to claim 1, characterized in that, The S200 synchronously acquires voice command streams, ambient light intensity data, motion sensor data, and GPS positioning data through sensors integrated into the AI ​​glasses; Input the voice command stream into the cloud-based collaborative speech recognition engine, run the lightweight speech recognition engine locally, and generate preliminary text commands; When the network is available, the audio stream is uploaded to the cloud in segments, the speech recognition engine is run to generate optimized text, and the optimized text is merged with the local results to generate text commands; The generated text commands are input into the natural language processing engine, and the output is a structured executable command containing: operation intent and parameter instructions.

4. The automated shooting method for AI glasses based on voice control according to claim 3, characterized in that, The natural language processing engine parses the operation intent through entity recognition and intent classification, generates executable commands, and binds the operation intent and parameter instructions in the executable commands to jointly execute the user commands. The rules for generating the operation intent are as follows: When the text command contains the configurable verb {record video}, output the shooting control command; When the text command contains the configurable verb {select keep delete}, output the media selection instruction; When the text command contains the configurable verb {editing acceleration}, output video editing instructions; When the text command contains the configurable verb {share, publish, upload}, output the social sharing command; The parameter command binding logic is as follows: shooting control commands are bound to parameter adjustment commands, media selection commands and video editing commands are bound to content processing commands, and social sharing commands are bound to the target platform identifier.

5. The automated shooting method for AI glasses based on voice control according to claim 1, characterized in that, The S300 responds to the operation intent in the executable command. When the intent is a shooting control command, it parses the parameter adjustment command and extracts the specific parameter values ​​of brightness, contrast, focal length, and exposure time. The parameter configuration interface of the vision processing engine is called to convert the extracted parameter values ​​into electrical signal instructions that can be recognized by the image sensor. The brightness parameter corresponds to adjusting the ISO sensitivity of the image sensor, the contrast parameter corresponds to adjusting the gamma curve of the image signal processor, and the focal length parameter achieves optical zoom by controlling the stepper motor of the camera. When taking a picture, the visual processing engine triggers the image sensor to perform a single-frame exposure. After the exposure is complete, the image data is transmitted to a temporary buffer. After autofocus and white balance calibration, a raw photo file in JPEG format is generated. During the recording operation, the visual processing engine controls the image sensor to continuously acquire frames and compress them in real time to generate the original video file, while recording the timestamp of each frame and the corresponding sensor parameters. After the original media file is generated, metadata tags such as shooting time, geographical location, and device model are automatically added and stored in the media storage and tagging unit.

6. The automated shooting method for AI glasses based on voice control according to claim 1, characterized in that, In S300, when the intent is a media selection instruction and it is directed at a photo, the intelligent photo selection analyzer reads a set of candidate photos from the media storage and tagging unit and extracts the image quality features of each photo, including the sharpness value, color saturation, and face detection results. Based on the filtering conditions in the content processing instructions, candidate photos are filtered and marked as initial targets. The scene type in the photos is identified by the sensor and matched with the scene description in the user instructions to further filter out the final target photos. Selected target photos are tagged with "Keep" and stored in the media storage and tagging unit. Unselected photos are tagged with "Pending Processing" and await further instructions from the user.

7. The automated shooting method for AI glasses based on voice control according to claim 1, characterized in that, In step S400, the intelligent video editor retrieves the time period information located in step S300 from the video segment index library to determine the range of the video segment to be edited. The video keyframes are used to identify specific time points such as transition points and motion peaks within the clip, which serve as reference nodes for editing and splicing. The original video is segmented and reassembled according to the editing requirements in the content processing instructions; After editing, a multi-track video file containing the main video track, audio track, and subtitle track is generated, allowing users to further adjust track parameters via voice commands.

8. A voice-controlled AI glasses automated shooting system, applicable to the voice-controlled AI glasses automated shooting method according to any one of claims 1-7, characterized in that, The system includes: a voice acquisition module, a command parsing module, a shooting control module, a media processing module, and a social sharing module; The voice acquisition module is integrated into the temples and nose pads of the AI ​​glasses. It includes a multi-channel directional microphone array and a real-time voice activity detection unit. It is used to continuously acquire ambient sound wave signals to form an initial voice stream, detect specific voice wake words through a local preset wake word recognition model, activate the command acquisition state and generate a voice command stream, and synchronously record audio timestamps. The instruction parsing module integrates a speech recognition engine and a natural language processing engine. After receiving the voice command stream, it combines real-time data obtained by the ambient light sensor, motion sensor and GPS module to generate text commands through edge-cloud collaborative recognition. After entity recognition and intent classification, it parses out the operation intent and parameter instructions, forms a structured executable command and completes instruction binding. The shooting control module includes a visual processing engine and an image sensor. When responding to shooting control commands, it parses parameter adjustment commands and converts them into electrical signals, controls the image sensor to perform photo and video recording operations, processes the image data, generates raw media files with metadata tags, and stores them in the media storage and tagging unit. The media processing module consists of an intelligent video editor and a media storage and tagging unit. When responding to a media selection instruction, it extracts photo quality features and filters target photos, locates the target time period of the video, and when responding to a video editing instruction, it performs video segmentation and recombination based on the location information to generate a multi-track edited video file. The social sharing module includes a platform interface adaptation unit and a tag generation unit. When responding to a social sharing command, it matches the target platform identifier with a preset API interface, performs format adaptation processing on the target media file, generates an environment tag through GPS data, and completes the media file upload after encryption and authentication.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory; The memory stores computer-readable instructions, which, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images