Voice control and content recognition method for controlling camera device

Through multimodal data fusion and high-dimensional space-time perception matrix processing, combined with deep learning and natural language processing, the problem of insufficient intelligent control and parameter adjustment of the camera device in complex scenarios is solved, and efficient object detection and scenario understanding are achieved, improving Shooting quality and user experience.

CN119071625AInactive Publication Date: 2024-12-03ZHONGKE FENGYUN (WUHAN) INFORMATION TECH CO LTD
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202411227165.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing camera devices are not intelligent and automated enough to meet complex scenes and diversified needs, cannot deeply analyze complex voice commands, lack the ability to comprehensive multimodal data analysis, resulting in unstable adjustment of shooting parameters and being unable to be in complex environments. Achieve intelligent decision-making.

Method used

Using multimodal data fusion technology, the generation and processing of high-dimensional space-time perception matrix, the mapping of affine geometric algebra space, and the high-dimensional index covariant tensor network, combined with deep learning and natural language processing, the precise analysis of voice commands and Dynamic adjustment of shooting parameters to improve target detection and scene understanding capabilities.

Benefits of technology

It realizes intelligent voice control, effective integration of multimodal data and dynamic parameter adjustment, improves the adaptability and shooting quality of the camera device in complex scenes, simplifies user operation process, and ensures the stability of shooting effects and user experience. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071625A_ABST
    Figure CN119071625A_ABST
Patent Text Reader

Abstract

The invention discloses a voice control and content recognition method for controlling a camera device, which comprises the following steps of: S1, receiving and analyzing a voice command of a user, and converting the voice command into a control instruction through a voice recognition module; s2, controlling the operation of the camera device according to the control instruction, and generating an image or a video; s3, acquiring image or video content in real time through an image acquisition module, encoding the image or video content into a high-dimensional space-time perception matrix, mapping the high-dimensional space-time perception matrix into an affine geometric algebraic space for target detection and scene understanding, and generating a content recognition vector; s4, based on the content identification vector, dynamically adjusting shooting parameters of the camera device to adapt to environmental changes; s5, when the specific target or scene is identified, automatically triggering a preset operation process; and S6, the content recognition result and the voice instruction are combined for processing, and an operation suggestion is generated or operation is automatically executed. According to the invention, intelligent control and efficient shooting of the camera device in a complex scene are realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of camera devices, and in particular to a voice controllable and content recognition method for controlling a camera device. Background Art

[0002] With the development of smart devices, cameras have become an indispensable tool in daily life and are widely used in security monitoring, mobile terminals, unmanned driving, smart homes and other fields. However, existing cameras still have many limitations in terms of intelligence, automation and user experience, making it difficult to meet complex scenarios and diverse needs.

[0003] Camera devices in the prior art usually rely on manual operation by users to achieve basic control functions through physical buttons or simple voice commands. Although voice recognition technology has made significant progress in recent years and supports basic voice command control, its functions are still relatively limited, mainly manifested in the following deficiencies:

[0004] First of all, existing voice control technologies usually only support basic command recognition and cannot deeply parse and understand complex voice commands. Users often need to clearly express specific commands during use, otherwise the device will find it difficult to execute accurately. When users want the camera to adjust shooting parameters or switch modes in complex scenes, existing voice recognition technology often cannot understand these requirements, let alone adjust the shooting settings in real time based on the scene. This causes users to frequently make manual adjustments during use, affecting the convenience and experience of operation.

[0005] Secondly, current cameras are relatively rudimentary in adjusting shooting parameters. Usually, they only adjust the exposure based on simple lighting conditions or adjust the focal length based on a fixed distance, without taking into account the complexity of multimodal data. Under different lighting conditions, in motion, or facing multi-target scenes, the camera lacks the ability to comprehensively analyze multimodal sensor data, making it difficult to dynamically adjust shooting parameters, and thus unable to guarantee the stability and quality of the shooting effect. This limitation in adjusting shooting parameters means that the camera often cannot adapt to the needs of different scenarios when dealing with rapidly changing environments.

[0006] Furthermore, although existing target detection and scene understanding technologies have made significant progress in the fields of image processing and computer vision, these technologies are often applied independently in camera devices, lacking an overall intelligent system that combines target detection with scene understanding and voice command processing. In traditional camera devices, target detection is usually only used to identify simple objects or faces, while scene understanding is limited to basic environmental judgment. This isolated processing method makes it impossible for the camera device to make intelligent decisions in complex scenes. When continuous tracking shooting is required in a specific scene, the camera devices of the prior art often have difficulty in automatically identifying and triggering the appropriate shooting mode, resulting in missed shots or poor shooting quality.

[0007] In addition, existing cameras lack an effective fusion mechanism when processing multimodal data. Usually, cameras can only rely on a single image data or a small amount of sensor information to adjust shooting parameters, but cannot effectively fuse data from different sensors to comprehensively judge environmental changes. This results in the camera being unable to accurately adjust shooting settings in complex and changing environments, and unable to achieve ideal shooting effects.

[0008] Therefore, how to provide a voice-controllable and content-recognition method for controlling a camera device is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0009] One purpose of the present invention is to propose a voice controllable and content recognition method for controlling a camera device. The present invention significantly improves the intelligence level and automation capability of the camera device by introducing advanced multimodal data fusion technology, generation and processing of high-dimensional spatiotemporal perception matrices, mapping of affine geometric algebraic spaces, and application of high-dimensional index covariant tensor networks. The method can automatically adjust shooting parameters according to the user's voice commands, optimize the camera effect in real time, and realize intelligent target detection and scene understanding in complex scenes. It has the advantages of high intelligence, convenient operation, and strong adaptability.

[0010] A method for controlling a camera device with voice control and content recognition according to an embodiment of the present invention comprises the following steps:

[0011] S1, receiving and parsing the user's voice command, and converting the voice command into a control instruction through a voice recognition module, wherein the control instruction includes an instruction for starting or stopping shooting, adjusting shooting parameters, and identifying specific scene operations;

[0012] S2. According to the control instructions, control the operation of the camera device, including starting or stopping video recording, capturing photos, adjusting focus, exposure compensation, white balance setting, and switching different shooting modes to generate images or videos;

[0013] S3, acquiring image or video content in real time through the image acquisition module of the camera device, encoding the image or video content extraction data into a high-dimensional spatiotemporal perception matrix, mapping the matrix to an affine geometric algebraic space, performing target detection through a high-dimensional index covariant tensor network, and then performing scene understanding through local projection topological transformation to generate a content recognition vector;

[0014] S4. Based on the result of the content recognition vector, the shooting parameter optimization module dynamically adjusts the exposure, focus and white balance settings of the camera device to adapt to the current lighting conditions and environmental changes;

[0015] S5. When the content recognition module recognizes a predetermined target object, a specific scene or an event, the camera device is automatically triggered to execute a preset operation process. The operation process includes continuous shooting, intelligent tracking shooting or automatic switching of shooting modes;

[0016] S6. Combine the content recognition results with the user's voice commands to generate operation suggestions with semantic understanding or automatically execute operations, and mark, classify and store the captured content according to user needs.

[0017] Optionally, the S1 specifically includes:

[0018] S11, the voice recognition module of the camera device receives the user's voice command and pre-processes the voice signal through a voice processing technology, wherein the pre-processing steps include noise suppression and signal enhancement;

[0019] S12, the processed voice data is input into a voice recognition model, and the voice recognition model uses a deep learning algorithm to convert the voice signal into text instructions;

[0020] S13, the generated text instruction is then passed to the instruction parsing module, in which the natural language processing algorithm is applied to parse the text instruction to extract the user's operation intention;

[0021] S14, after analyzing the operation intention, the camera device generates a corresponding control instruction;

[0022] S15. The generated control instruction is transmitted to the execution module of the camera device.

[0023] Optionally, the S2 specifically includes:

[0024] S21, after receiving and parsing the user's control signal, judging whether to start or stop shooting of the camera device through the hidden Markov model:

[0025]

[0026] Among them, P start / stopis the probability of starting or stopping shooting, θ is the parameter set of the hidden Markov model, x t is the time series data, γ t is the weight coefficient of state transfer, T is the total length of the time series;

[0027] S22, after determining that shooting is started, the camera device enters a focus adjustment phase, and uses a multi-scale geometric analysis algorithm to adjust the focus;

[0028] S23, after the focal length is adjusted, the camera device automatically adjusts the exposure parameters, using a nonlinear exposure compensation method based on topological data analysis;

[0029] S24. After the exposure adjustment is completed, the camera device performs white balance adjustment according to the color temperature information of the image, and uses random matrix theory to perform color temperature correction to generate the final image or video:

[0030]

[0031] Among them, W base is the basic white balance value, λ i is the eigenvalue of the image covariance matrix, and N is the total number of eigenvalues.

[0032] Optionally, the S3 specifically includes:

[0033] S31, the camera device obtains real-time image or video content through multimodal sensor fusion technology, and jointly encodes the image data with other sensor data such as acceleration, temperature, and illumination to generate a high-dimensional spatiotemporal perception matrix I sp (x,y,t;α,β,ζ):

[0034]

[0035] Among them, x and y are the spatial coordinates of the image, t is the time dimension, and α j and β k are respectively related to the spatial data matrix S j (x,y) and time-dependent data function T k (t) is the weight parameter related to the nonlinear mapping, ζ is the control parameter of the nonlinear mapping, I(x,y,t) represents the spatiotemporal representation of the image data, which varies with space (x,y) and time t, S j (x, y) is the spatial data matrix, describing the spatial distribution characteristics of the multimodal sensor, T k (t) is the time-related data function, reflecting the change of sensor data over time, α j and β k is the weight parameter for spatial and temporal related data, is a nonlinear mapping operator, which is affected by the control parameter ζ, and λ is a regularization parameter used to balance the influence of different modal data;

[0036] S32, generated spatiotemporal perception matrix I sp It is then mapped to the affine geometric algebra space. The operations of affine geometric algebra capture the complex geometric relationships between multimodal data and optimize the nonlinear relationships in the spatiotemporal perception matrix:

[0037]

[0038] in, is the space-time perception matrix, I sp Representation in affine geometric algebraic space, is the affine transformation matrix, describing the affine geometric operation on the space, is a spinor used to process the rotation relationship of multimodal data. and is the corresponding affine transformation basis function, which is used to map spatial and temporal data into affine geometric algebraic space;

[0039] S33. In affine geometric algebraic space, do we use high-dimensional index covariant tensor networks for target detection and recognition? Through operations in affine geometric algebraic space, complex geometric relationships in affine space are encoded into the target detection model, optimizing the model's robustness to various transformations:

[0040]

[0041] in, is the covariant tensor response of the target category C, T lm is a high-dimensional index covariant tensor, where the indices l and m represent the different dimensions of the tensor. is a high-dimensional index function used to adjust the response of the tensor network, It is the basis function of the spatiotemporal perception matrix in the affine geometric algebra space, which is used to combine multimodal data for target detection;

[0042] S34. After the target is detected in the affine geometric algebraic space, the scene is further mapped to the non-Euclidean space. The scene is understood through local projective topological transformation. The detected target and environmental context information are projected onto the local non-Euclidean manifold to capture the complex topological structure in the scene.

[0043] S35. The camera device combines the results of target detection and scene understanding to generate a comprehensive nonlinear hybrid content recognition vector R mix , a vector containing the position of the target in spacetime and affine geometric algebraic space (x c ,y c ,tc ), and also includes the scene representation S after local projection topological transformation proj and semantic information proj ).

[0044] Optionally, the S4 specifically includes:

[0045] S41, after the content recognition module completes the target detection and scene understanding, the camera device will use the nonlinear hybrid content recognition vector R mix The scene information S in proj and the target position x c ,y c ,t c As input, the shooting parameter optimization module is started, which calculates the priority weights ωx,y,t of different areas in the image. The weights are determined by the scene complexity and the importance of the target:

[0046] ωx,y,t=PriorityWeightS proj ,x c ,y c ,t c ;

[0047] The PriorityWeight function is based on the complexity of the scene S proj and the importance of goals x c ,y c ,t c To determine the priority weights;

[0048] S42. According to the priority weight ωx,y,t, the shooting parameter optimization module uses a nonlinear exposure compensation model to adjust the exposure parameter Ex,y,t. The model dynamically adjusts the exposure value of each pixel based on the scene brightness distribution Lx,y,t and the priority weight:

[0049]

[0050] Among them, E0 is the basic exposure value, Lx,y,t is the pixel brightness distribution, is a nonlinear mapping function of brightness, and the nonlinear exposure compensation model function combines these parameters to calculate the final exposure value;

[0051] S43. To further optimize the shooting effect, the shooting parameter optimization module will adjust the focal length Zx,y,t through the curvature feedback model. The curvature feedback model optimizes the focal length distribution by combining the curvature feedback of the focal length Zx,y,t and the weight response θx,y,t:

[0052]

[0053] Among them, κZx,y,t,θx,y,t are the curvature feedback results, η is the adjustment parameter, and θx,y,t is the weight response calculated by combining the exposure parameters and weights mentioned above;

[0054] S44, the processed exposure parameters Ex,y,t and focal length distribution Zx,y,t are input to the white balance adjustment module, and the white balance adjustment module dynamically adjusts the white balance parameters Wx,y,t based on the consideration of color fidelity Cx,y,t and brightness consistency through the model WhiteBalanceAdjust:

[0055]

[0056] Among them, W0 is the basic white balance value, Cx, y, t is the color fidelity, and the WhiteBalanceAdjust function combines the exposure parameters and color fidelity to calculate the final white balance parameters Wx, y, t;

[0057] S45. The curvature feedback model shooting parameter optimization module updates the shooting settings of the camera device in real time according to the optimized exposure, focal length and white balance parameters, and the optimized parameters are applied to the rendering process of the image or video content.

[0058] Optionally, the S5 specifically includes:

[0059] S51: After the content recognition and shooting parameter optimization are completed, the camera device starts the analysis module to calculate the target's motion trajectory and scene changes according to the recognized target object and scene information, and calculates the target position x c ,y c ,t c The rate of change of Δx c ,y c ,t c , and combined with the scene recognition result S proj To evaluate the target's movement trend;

[0060] S52, based on the analysis result of the motion trend, by judging the dynamic changes of the current scene, decide whether to trigger the shooting operation process, continuous shooting, intelligent tracking shooting or automatic switching shooting mode. Once the operation is triggered, the camera device will c ,y c ,t c With the optimized shooting parameters, focal length Zx,y,t and priority weight ωx,y,t, the shooting settings are adjusted dynamically;

[0061] S53, in the intelligent tracking shooting mode, the camera device uses the target's motion trend information and the previously optimized shooting parameters to adjust the focal length Z in real time trackx, y, t and the position of the viewfinder F track x,y,t;

[0062] S54: During continuous shooting or automatic switching of shooting modes, the camera device detects the current scene recognition result and the motion trend Δx. c ,y c ,t c , dynamically switch the shooting mode;

[0063] S55, in order to continuously optimize the shooting effect, the camera device continuously adjusts the shooting parameters through the feedback loop function while executing the preset operation process, and the feedback loop calculates the time derivative of the target motion change rate in real time. To dynamically update the shooting parameters P adjust x,y,t.

[0064] Beneficial Effects

[0065] 1. Intelligent voice control: The present invention uses deep learning algorithms and natural language processing technology to achieve accurate recognition and analysis of complex voice commands. Through the adaptive voice recognition module, users can naturally issue commands, and the camera device can intelligently understand and execute complex operations, start or stop shooting, dynamically adjust shooting parameters, and intelligently switch shooting modes, which greatly simplifies the user operation process and improves the convenience and accuracy of operation.

[0066] 2. Multimodal data fusion and dynamic shooting parameter adjustment: By introducing multimodal sensor fusion technology, the camera device can comprehensively process multiple sensor data such as image, acceleration, temperature, and light to generate a high-dimensional spatiotemporal perception matrix. Based on this matrix, the camera device can adjust the focus, exposure, and white balance shooting parameters in real time under different lighting conditions and complex environments to ensure the best shooting effect in various scenes. This intelligent parameter adjustment avoids the problem of unstable shooting quality caused by scene changes in traditional equipment.

[0067] 3. Efficient target detection and scene understanding: The present invention significantly enhances the target detection and scene understanding capabilities of the camera device through the mapping of affine geometric algebraic space and the application of high-dimensional index covariant tensor networks. The camera device can accurately identify target objects and scenes in complex and changing environments, generate comprehensive nonlinear mixed content recognition vectors, and improve adaptability and processing efficiency to complex scenes. This innovation enables the camera device to not only recognize simple static scenes, but also intelligently track targets in dynamic scenes, automatically adjust the shooting mode, and ensure that key scenes are not missed.

[0068] 4. Automated operation process and real-time feedback: When a predetermined target or specific scene is detected, the camera device can automatically trigger the preset operation process, continuous shooting or intelligent tracking shooting. Through the application of real-time feedback loops, the system can dynamically adjust shooting parameters to cope with changes in targets or scenes.

[0069] 5. Semantic understanding and multimedia content management: Finally, the camera device combines the content recognition results with the user's voice commands to generate semantically understood operation suggestions or automatically execute operations. This function supports intelligent tagging, classified storage, or generation of multimedia content output with additional information for the captured content, making it more convenient and efficient for users to subsequently process and manage the captured content, improving the overall user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0071] Figure 1 A summary diagram of a voice control and content recognition method for controlling a camera device proposed by the present invention;

[0072] Figure 2 The figure of claim 4 of the method for controlling a camera device with voice control and content recognition proposed by the present invention;

[0073] Figure 3 The figure of claim 5 of the voice control and content recognition method for controlling a camera device proposed by the present invention. DETAILED DESCRIPTION

[0074] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0075] refer to Figure 1-3 , a method for controlling a camera device with voice control and content recognition, comprising the following steps:

[0076] S1, receiving and parsing the user's voice command, and converting the voice command into a control instruction through a voice recognition module, wherein the control instruction includes an instruction for starting or stopping shooting, adjusting shooting parameters, and identifying specific scene operations;

[0077] S2. According to the control instructions, control the operation of the camera device, including starting or stopping video recording, capturing photos, adjusting focus, exposure compensation, white balance setting, and switching different shooting modes to generate images or videos;

[0078] S3, acquiring image or video content in real time through the image acquisition module of the camera device, encoding the image or video content extraction data into a high-dimensional spatiotemporal perception matrix, mapping the matrix to an affine geometric algebraic space, performing target detection through a high-dimensional index covariant tensor network, and then performing scene understanding through local projection topological transformation to generate a content recognition vector;

[0079] S4. Based on the result of the content recognition vector, the shooting parameter optimization module dynamically adjusts the exposure, focus and white balance settings of the camera device to adapt to the current lighting conditions and environmental changes;

[0080] S5. When the content recognition module recognizes a predetermined target object, a specific scene or an event, the camera device is automatically triggered to execute a preset operation process. The operation process includes continuous shooting, intelligent tracking shooting or automatic switching of shooting modes;

[0081] S6. Combine the content recognition results with the user's voice commands to generate operation suggestions with semantic understanding or automatically execute operations, and mark, classify and store the captured content according to user needs.

[0082] In this implementation, S1 specifically includes:

[0083] S11, the voice recognition module of the camera device receives the user's voice command and pre-processes the voice signal through a voice processing technology, wherein the pre-processing steps include noise suppression and signal enhancement;

[0084] S12, the processed voice data is input into a voice recognition model, and the voice recognition model uses a deep learning algorithm to convert the voice signal into text instructions;

[0085] S13, the generated text instruction is then passed to the instruction parsing module, in which the natural language processing algorithm is applied to parse the text instruction to extract the user's operation intention;

[0086] S14, after analyzing the operation intention, the camera device generates a corresponding control instruction;

[0087] S15. The generated control instruction is transmitted to the execution module of the camera device.

[0088] In this implementation, S2 specifically includes:

[0089] S21, after receiving and parsing the user's control signal, judging whether to start or stop shooting of the camera device through the hidden Markov model:

[0090]

[0091] Among them, P start / stopis the probability of starting or stopping shooting, θ is the parameter set of the hidden Markov model, x t is the time series data, γ t is the weight coefficient of state transfer, T is the total length of the time series;

[0092] S22, after determining that shooting is started, the camera device enters a focus adjustment phase, and uses a multi-scale geometric analysis algorithm to adjust the focus;

[0093] S23, after the focal length is adjusted, the camera device automatically adjusts the exposure parameters, using a nonlinear exposure compensation method based on topological data analysis;

[0094] S24. After the exposure adjustment is completed, the camera device performs white balance adjustment according to the color temperature information of the image, and uses random matrix theory to perform color temperature correction to generate the final image or video:

[0095]

[0096] Among them, W base is the basic white balance value, λ i is the eigenvalue of the image covariance matrix, and N is the total number of eigenvalues.

[0097] In this implementation, S3 specifically includes:

[0098] S31, the camera device obtains real-time image or video content through multimodal sensor fusion technology, and jointly encodes the image data with other sensor data such as acceleration, temperature, and illumination to generate a high-dimensional spatiotemporal perception matrix I sp (x,y,t;α,β,ζ):

[0099]

[0100] Among them, x and y are the spatial coordinates of the image, t is the time dimension, and α j and β k are respectively related to the spatial data matrix S j (x,y) and time-dependent data function T k (t) is the weight parameter related to the nonlinear mapping, ζ is the control parameter of the nonlinear mapping, I(x,y,t) represents the spatiotemporal representation of the image data, which varies with space (x,y) and time t, S j (x, y) is the spatial data matrix, describing the spatial distribution characteristics of the multimodal sensor, T k (t) is the time-related data function, reflecting the change of sensor data over time, α j and β k is the weight parameter for spatial and temporal related data, is a nonlinear mapping operator, which is affected by the control parameter ζ, and λ is a regularization parameter used to balance the influence of different modal data;

[0101] S32, generated spatiotemporal perception matrix I sp It is then mapped to the affine geometric algebra space. The operations of affine geometric algebra capture the complex geometric relationships between multimodal data and optimize the nonlinear relationships in the spatiotemporal perception matrix:

[0102]

[0103] in, is the space-time perception matrix, I sp Representation in affine geometric algebraic space, is the affine transformation matrix, describing the affine geometric operation on the space, is a spinor used to process the rotation relationship of multimodal data. and is the corresponding affine transformation basis function, which is used to map spatial and temporal data into affine geometric algebraic space;

[0104] S33. In affine geometric algebraic space, do we use high-dimensional index covariant tensor networks for target detection and recognition? Through operations in affine geometric algebraic space, complex geometric relationships in affine space are encoded into the target detection model, optimizing the model's robustness to various transformations:

[0105]

[0106] in, is the covariant tensor response of the target category C, T lm is a high-dimensional index covariant tensor, where the indices l and m represent the different dimensions of the tensor. is a high-dimensional index function used to adjust the response of the tensor network, It is the basis function of the spatiotemporal perception matrix in the affine geometric algebra space, which is used to combine multimodal data for target detection;

[0107] S34. After the target is detected in the affine geometric algebraic space, the scene is further mapped to the non-Euclidean space. The scene is understood through local projective topological transformation. The detected target and environmental context information are projected onto the local non-Euclidean manifold to capture the complex topological structure in the scene.

[0108] S35. The camera device combines the results of target detection and scene understanding to generate a comprehensive nonlinear hybrid content recognition vector R mix , a vector containing the position of the target in spacetime and affine geometric algebraic space (x c ,y c ,tc ), and also includes the scene representation S after local projection topological transformation proj and semantic information proj ).

[0109] In this implementation, S4 specifically includes:

[0110] S41, after the content recognition module completes the target detection and scene understanding, the camera device will use the nonlinear hybrid content recognition vector R mix The scene information S in proj and the target position x c ,y c ,t c As input, the shooting parameter optimization module is started, which calculates the priority weights ωx,y,t of different areas in the image. The weights are determined by the scene complexity and the importance of the target:

[0111] ωx,y,t=PriorityWeightS proj ,x c ,y c ,t c ;

[0112] The PriorityWeight function is based on the complexity of the scene S proj and the importance of goals x c ,y c ,t c To determine the priority weights;

[0113] S42. According to the priority weight ωx,y,t, the shooting parameter optimization module uses a nonlinear exposure compensation model to adjust the exposure parameter Ex,y,t. The model dynamically adjusts the exposure value of each pixel based on the scene brightness distribution Lx,y,t and the priority weight:

[0114]

[0115] Among them, E0 is the basic exposure value, Lx,y,t is the pixel brightness distribution, is a nonlinear mapping function of brightness, and the nonlinear exposure compensation model function combines these parameters to calculate the final exposure value;

[0116] S43. To further optimize the shooting effect, the shooting parameter optimization module will adjust the focal length Zx,y,t through the curvature feedback model. The curvature feedback model optimizes the focal length distribution by combining the curvature feedback of the focal length Zx,y,t and the weight response θx,y,t:

[0117]

[0118] Among them, κZx,y,t,θx,y,t are the curvature feedback results, η is the adjustment parameter, and θx,y,t is the weight response calculated by combining the exposure parameters and weights mentioned above;

[0119] S44, the processed exposure parameters Ex,y,t and focal length distribution Zx,y,t are input to the white balance adjustment module, and the white balance adjustment module dynamically adjusts the white balance parameters Wx,y,t based on the consideration of color fidelity Cx,y,t and brightness consistency through the model WhiteBalanceAdjust:

[0120]

[0121] Among them, W0 is the basic white balance value, Cx, y, t is the color fidelity, and the WhiteBalanceAdjust function combines the exposure parameters and color fidelity to calculate the final white balance parameters Wx, y, t;

[0122] S45. The curvature feedback model shooting parameter optimization module updates the shooting settings of the camera device in real time according to the optimized exposure, focal length and white balance parameters, and the optimized parameters are applied to the rendering process of the image or video content.

[0123] In this implementation manner, S5 specifically includes:

[0124] S51: After the content recognition and shooting parameter optimization are completed, the camera device starts the analysis module to calculate the target's motion trajectory and scene changes according to the recognized target object and scene information, and calculates the target position x c ,y c ,t c The rate of change of Δx c ,y c ,t c , and combined with the scene recognition result S proj To evaluate the target's movement trend;

[0125] S52, based on the analysis result of the motion trend, by judging the dynamic changes of the current scene, decide whether to trigger the shooting operation process, continuous shooting, intelligent tracking shooting or automatic switching shooting mode. Once the operation is triggered, the camera device will c ,y c ,t c With the optimized shooting parameters, focal length Zx,y,t and priority weight ωx,y,t, the shooting settings are adjusted dynamically;

[0126] S53, in the intelligent tracking shooting mode, the camera device uses the target's motion trend information and the previously optimized shooting parameters to adjust the focal length Z in real time trackx, y, t and the position of the viewfinder F track x,y,t;

[0127] S54: During continuous shooting or automatic switching of shooting modes, the camera device detects the current scene recognition result and the motion trend Δx. c ,y c ,t c , dynamically switch the shooting mode;

[0128] S55, in order to continuously optimize the shooting effect, the camera device continuously adjusts the shooting parameters through the feedback loop function while executing the preset operation process, and the feedback loop calculates the time derivative of the target motion change rate in real time. To dynamically update the shooting parameters P adjust x,y,t.

[0129] Example

[0130] In order to verify the feasibility of the present invention in implementation, the present invention is applied to a smart home environment in a certain city, where users hope to achieve comprehensive recording and management of family life through smart camera devices. The home includes multiple rooms, corridors, kitchens and courtyards. The user hopes that the camera device can automatically adjust the shooting parameters according to the daily activities of family members, and record important life fragments, such as family dinners, pets playing, and visitors. At the same time, when the camera device detects special scenes, when children are playing in the courtyard and family members enter the kitchen to prepare meals, it can automatically track and record important moments. In such scenes, existing ordinary camera devices are often difficult to achieve intelligent and accurate recording due to changes in light, complex home layouts, and diverse activity content.

[0131] In order to meet the recording needs in home scenarios, this embodiment applies the voice controllable and content recognition method of the present invention:

[0132] The user sends a voice command to the camera at home, "Start recording family activities". The camera receives and processes the command through the voice recognition module, converts the voice signal into a control signal, and instructs the camera to start the round-the-clock family life recording function.

[0133] In daily family activities, the camera device uses multimodal sensors such as light sensors, temperature sensors, and motion detectors to obtain real-time home environment data and generate a high-dimensional spatiotemporal perception matrix. When the camera device detects that daylight is coming in from the window, causing changes in indoor light, it automatically adjusts the exposure parameters to prevent overexposure of the image; when the light gets darker at night, it increases the exposure time and white balance settings to ensure the brightness and color reproduction of the image. At the same time, the camera device uses a curvature feedback model to adjust the focal length to ensure that the camera device can clearly capture every corner of the home.

[0134] While acquiring video content, the camera device analyzes and identifies the daily activities of family members, pets and visitors in real time. When a family member is detected entering the living room, the camera device accurately identifies the person's position and automatically adjusts the shooting angle through affine geometric algebraic space mapping and high-dimensional index covariant tensor network to ensure that clear and natural family interaction scenes are captured. In addition, it can recognize the visitor's facial information and determine whether he or she is a family member or a common visitor. When an important moment family gathering or birthday celebration is detected, high-definition images or videos are automatically recorded to help users preserve precious family memories.

[0135] When detecting the movement of family members in important life activities, the play of pets, or the arrival of visitors, the camera device will automatically switch to intelligent tracking mode, adjusting the focus and frame position in real time to ensure that the subject of the activity is always within the camera's field of view. Through local projection topological transformation, it can more deeply understand the structure of the scene, recognize family members cooking in the kitchen or walking in the yard, and automatically generate records. These automatically captured contents can not only be stored in real time, but also generate organized multimedia content for users, helping them to easily review and share the beautiful moments of family life.

[0136] In the one-month family life recording experiment, smart cameras were installed at key locations such as the home entrance, living room, kitchen and courtyard. The specific data are as follows:

[0137] Table 1: Recording effect data of smart camera device in daytime and nighttime environments

[0138]

[0139] Table 2: Detection and recording data of key life events by smart camera devices

[0140]

[0141] Table 3: Effect data of the intelligent camera device's courtyard intelligent tracking function

[0142] Function Intelligent tracking effect in the courtyard Tracking accuracy 96% Target out of sight none

[0143] From the above table, it can be concluded that the smart camera device performs well in different light conditions during the day and at night. During the day, the exposure parameters were automatically adjusted in an environment with large light changes, and about 500 hours of video content were recorded, with a clarity of 97%, and an accuracy of 99% in capturing important life fragments. At night, by increasing the exposure time and adjusting the white balance parameters, the brightness and clarity of the image were ensured, and 300 hours of video content were recorded, with a clarity of 94%, and an accuracy of 98% in capturing important life fragments. In addition, 8 family dinners and 5 important visitors were successfully detected and recorded, and the automatic recording mode was started within 2 seconds. It also recognized 20 pet activities and 15 operations of family members in the kitchen, generating high-definition images and videos. At the same time, the intelligent tracking function in the courtyard enables the activities of family members and pets to be accurately captured with a tracking accuracy of 96%, and no target is out of sight during the process.

[0144] The above data and experimental results prove that the voice control and content recognition method described in the present invention significantly improves the intelligence and automation level of the camera device in the smart home environment. It can adapt to various changes in the home environment, adjust the shooting parameters in real time, and provide an efficient life recording solution through intelligent detection and tracking functions. This embodiment shows the significant advantages of the present invention in solving the problems of insufficient intelligence of the camera device, inaccurate parameter adjustment, and untimely capture of important life fragments in the prior art, providing a more intelligent and convenient way for family life recording.

[0145] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for controlling a camera device with voice control and content recognition, characterized in that: The steps include: S1, receiving and parsing the user's voice command, and converting the voice command into a control instruction through a voice recognition module, wherein the control instruction includes an instruction for starting or stopping shooting, adjusting shooting parameters, and identifying specific scene operations; S2. According to the control instructions, control the operation of the camera device, including starting or stopping video recording, capturing photos, adjusting focus, exposure compensation, white balance setting, and switching different shooting modes to generate images or videos; S3, acquiring image or video content in real time through the image acquisition module of the camera device, encoding the image or video content extraction data into a high-dimensional spatiotemporal perception matrix, mapping the matrix to an affine geometric algebraic space, performing target detection through a high-dimensional index covariant tensor network, and then performing scene understanding through local projection topological transformation to generate a content recognition vector; S4. Based on the result of the content recognition vector, the shooting parameter optimization module dynamically adjusts the exposure, focus and white balance settings of the camera device to adapt to the current lighting conditions and environmental changes; S5. When the content recognition module recognizes a predetermined target object, a specific scene or an event, the camera device is automatically triggered to execute a preset operation process. The operation process includes continuous shooting, intelligent tracking shooting or automatic switching of shooting modes; S6. Combine the content recognition results with the user's voice commands to generate operation suggestions with semantic understanding or automatically execute operations, and mark, classify and store the captured content according to user needs.

2. A method for controlling a camera device with voice control and content recognition according to claim 1, characterized in that: The S1 specifically includes: S11, the voice recognition module of the camera device receives the user's voice command and pre-processes the voice signal through a voice processing technology, wherein the pre-processing steps include noise suppression and signal enhancement; S12, the processed voice data is input into a voice recognition model, and the voice recognition model uses a deep learning algorithm to convert the voice signal into text instructions; S13, the generated text instruction is then passed to the instruction parsing module, in which the natural language processing algorithm is applied to parse the text instruction to extract the user's operation intention; S14, after analyzing the operation intention, the camera device generates a corresponding control instruction; S15. The generated control instruction is transmitted to the execution module of the camera device.

3. The method for voice control and content recognition for controlling a camera device according to claim 1, characterized in that: The S2 specifically includes: S21, after receiving and parsing the user's control signal, judging whether to start or stop shooting of the camera device through the hidden Markov model: Among them, P start / stop is the probability of starting or stopping shooting, θ is the parameter set of the hidden Markov model, x t is the time series data, γ t is the weight coefficient of state transfer, T is the total length of the time series; S22, after determining that shooting is started, the camera device enters a focus adjustment phase, and uses a multi-scale geometric analysis algorithm to adjust the focus; S23, after the focal length is adjusted, the camera device automatically adjusts the exposure parameters, using a nonlinear exposure compensation method based on topological data analysis; S24. After the exposure adjustment is completed, the camera device performs white balance adjustment according to the color temperature information of the image, and uses random matrix theory to perform color temperature correction to generate the final image or video: Among them, W base is the basic white balance value, λ i is the eigenvalue of the image covariance matrix, and N is the total number of eigenvalues.

4. The method for voice control and content recognition for controlling a camera device according to claim 1, characterized in that: The S3 specifically includes: S31, the camera device obtains real-time image or video content through multimodal sensor fusion technology, and jointly encodes the image data with other sensor data such as acceleration, temperature, and illumination to generate a high-dimensional spatiotemporal perception matrix I sp (x,y,t;α,β,ζ): Among them, x and y are the spatial coordinates of the image, t is the time dimension, and α j and β k are respectively related to the spatial data matrix S j (x,y) and time-dependent data function T k (t) is the weight parameter related to the nonlinear mapping, ζ is the control parameter of the nonlinear mapping, I(x,y,t) represents the spatiotemporal representation of the image data, which varies with space (x,y) and time t, S j (x, y) is the spatial data matrix, describing the spatial distribution characteristics of the multimodal sensor, T k (t) is the time-related data function, reflecting the change of sensor data over time, α j and β k is the weight parameter for spatial and temporal related data, is a nonlinear mapping operator, which is affected by the control parameter ζ, and λ is a regularization parameter used to balance the influence of different modal data; S32, generated spatiotemporal perception matrix I sp It is then mapped to the affine geometric algebra space. The operations of affine geometric algebra capture the complex geometric relationships between multimodal data and optimize the nonlinear relationships in the spatiotemporal perception matrix: in, is the space-time perception matrix, I sp Representation in affine geometric algebraic space, is the affine transformation matrix, describing the affine geometric operation on the space, is a spinor used to process the rotation relationship of multimodal data. and is the corresponding affine transformation basis function, which is used to map spatial and temporal data into affine geometric algebraic space; S33. In affine geometric algebraic space, do we use high-dimensional index covariant tensor networks for target detection and recognition? Through operations in affine geometric algebraic space, complex geometric relationships in affine space are encoded into the target detection model, optimizing the model's robustness to various transformations: in, is the covariant tensor response of the target category C, T lm is a high-dimensional index covariant tensor, where the indices l and m represent the different dimensions of the tensor. is a high-dimensional index function used to adjust the response of the tensor network, It is the basis function of the spatiotemporal perception matrix in the affine geometric algebra space, which is used to combine multimodal data for target detection; S34. After the target is detected in the affine geometric algebraic space, the scene is further mapped to the non-Euclidean space. The scene is understood through local projective topological transformation. The detected target and environmental context information are projected onto the local non-Euclidean manifold to capture the complex topological structure in the scene. S35. The camera device combines the results of target detection and scene understanding to generate a comprehensive nonlinear hybrid content recognition vector R mix , a vector containing the position of the target in spacetime and affine geometric algebraic space (x c ,y c ,t c ), and also includes the scene representation S after local projection topological transformation proj and semantic information proj ).

5. The method for voice control and content recognition for controlling a camera device according to claim 1, characterized in that: The S4 specifically includes: S41, after the content recognition module completes the target detection and scene understanding, the camera device will use the nonlinear hybrid content recognition vector R mix The scene information S in proj and the target position (x c ,y c ,t c ) as input, start the shooting parameter optimization module, the module calculates the priority weights ω(x, y, t) of different areas in the image, the weights are determined by the complexity of the scene and the importance of the target: ω(x,y,t)=PriorityWeight(S proj ,(x c ,y c ,t c )); The PriorityWeight function is based on the complexity of the scene S proj and the importance of the goal (x c ,y c ,t c ) to determine the priority weight; S42. According to the priority weight ω(x, y, t), the shooting parameter optimization module uses a nonlinear exposure compensation model to adjust the exposure parameter E(x, y, t). The model dynamically adjusts the exposure value of each pixel based on the scene brightness distribution L(x, y, t) and the priority weight: Among them, E0 is the basic exposure value, L(x,y,t) is the pixel brightness distribution, is a nonlinear mapping function of brightness, and the nonlinear exposure compensation model function combines these parameters to calculate the final exposure value; S43. To further optimize the shooting effect, the shooting parameter optimization module adjusts the focal length Z(x, y, t) through the curvature feedback model. The curvature feedback model optimizes the focal length distribution by combining the curvature feedback of the focal length Z(x, y, t) and the weighted response θ(x, y, t): Among them, κ(Z(x,y,t),θ(x,y,t)) is the curvature feedback result, η is the adjustment parameter, and θ(x,y,t) is the weight response calculated by combining the exposure parameter and weight mentioned above; S44, the processed exposure parameters E(x, y, t) and focal length distribution Z(x, y, t) are input to the white balance adjustment module, and the white balance adjustment module dynamically adjusts the white balance parameters W(x, y, t) based on the consideration of color fidelity C(x, y, t) and brightness consistency through the model WhiteBalanceAdjust: Among them, W0 is the basic white balance value, C(x,y,t) is the color fidelity, and the WhiteBalanceAdjust function combines the exposure parameters and color fidelity to calculate the final white balance parameters W(x,y,t); S45. The curvature feedback model shooting parameter optimization module updates the shooting settings of the camera device according to the optimized exposure, focal length and white balance parameters.

6. The method for voice control and content recognition for controlling a camera device according to claim 1, characterized in that: The S5 specifically includes: S51: After the content recognition and shooting parameter optimization are completed, the camera device starts the analysis module to calculate the target's motion trajectory and scene changes according to the recognized target object and scene information. c ,y c ,t c ) is the rate of change Δ(x c ,y c ,t c ), and combined with the scene recognition result S proj To evaluate the target's movement trend; S52, based on the analysis result of the motion trend, by judging the dynamic changes of the current scene, decide whether to trigger the shooting operation process, continuous shooting, intelligent tracking shooting or automatic switching shooting mode. Once the operation is triggered, the camera device will c ,y c ,t c ) and the optimized shooting parameters, focal length Z(x,y,t) and priority weight ω(x,y,t), adjust the shooting settings; S53: In the intelligent tracking shooting mode, the camera device uses the target's motion trend information and the previously optimized shooting parameters to adjust the focal length Z. track (x, y, t) and the position of the viewfinder F track (x,y,t); S54: During continuous shooting or automatic switching of shooting modes, the camera device detects the current scene recognition result and the motion trend Δ(x c ,y c ,t c ), dynamically switch the shooting mode; S55, in order to continuously optimize the shooting effect, the camera device continuously adjusts the shooting parameters through the feedback loop function while executing the preset operation process, and the feedback loop calculates the time derivative of the target motion change rate in real time. To dynamically update the shooting parameters P adjust (x,y,t).

Citation Information

Cited By

  • Following shot method and system based on smart big data

    CN119233074A

  • Shooting method, wearable device and readable storage medium

    CN119815171A

  • Photographing method, wearable device, and readable storage medium

    CN119815171B

  • Mouse gait analysis method based on following shooting

    CN119992652A

  • A mouse gait analysis method based on follow-up filming

    CN119992652B