A computer-TV integrated machine control method, system and device

Abstract

The present invention relates to the field of computer television image communication technology, and in particular to a computer television integrated machine control method, system and device. The method comprises the following steps: through multi-view image acquisition and processing, a three-dimensional scene grid is generated and scene element analysis is performed to form a scene element set, then the user's limbs are chased to generate a user limb image set, and the environmental tone is restored at the same time, the display content of the integrated machine is obtained and the content semantic analysis is performed, the interaction intensity is calculated to obtain content interaction intensity data, the user's body language is inferred and the user intention is predicted in combination with the interaction intensity data, based on the predicted user operation intention, the display content of the integrated machine is adjusted, and simulated adjustment content is generated, and the composition is optimized through the scene element set to generate updated display content. The present invention realizes a more accurate and efficient computer television integrated machine control method.

Description

Technical Field

[0001] The present invention relates to the technical field of computer television image communication, and in particular to a computer television integrated machine control method, system and device. Background Art

[0002] People's expectations for the interaction of integrated devices such as computers and televisions are gradually shifting from traditional remote controls and button operations to more natural and intuitive methods, especially the introduction of body language and gesture control, which allows users to interact with devices in a more flexible way. However, most of the current interactive systems lack accurate recognition of complex body movements, resulting in a poor user experience during use. Existing technologies are insufficient in environmental perception and understanding of user intentions. Many systems are unable to capture user movements and emotional changes in real time, which affects the device's response speed to user needs. Especially when multiple people use it at the same time, it is difficult for the system to effectively distinguish the intentions of different users, resulting in inaccurate information transmission, further reducing user satisfaction and device usage efficiency. Summary of the invention

[0003] Based on this, it is necessary to provide a computer-TV all-in-one control method, system and device to solve at least one of the above technical problems.

[0004] To achieve the above object, a computer-TV all-in-one control method comprises the following steps:

[0005] Step S1: collecting multi-view images of the integrated machine; performing spatial grid modeling on the multi-view images of the integrated machine to obtain a three-dimensional scene grid; performing scene element analysis on the three-dimensional scene grid to generate a scene element set;

[0006] Step S2: tracking the user's limbs on the multi-view images of the all-in-one device to generate a user limb image set; performing environmental simulation restoration on the scene element set to obtain the use environment tone;

[0007] Step S3: acquiring the display content of the all-in-one machine; performing content semantic analysis on the display content of the all-in-one machine to obtain content semantic data; calculating the interaction intensity of the content semantic data according to the tone of the usage environment to generate content interaction intensity data;

[0008] Step S4: inferring the body language of the user's body image set based on the tone of the usage environment to obtain the user's body language; predicting the user's intention based on the user's body language according to the content interaction intensity data to generate a predicted user operation intention;

[0009] Step S5: adjusting the display content of the all-in-one machine based on the predicted user operation intention to obtain simulated adjustment content; optimizing the composition of the simulated adjustment content according to the scene element set to generate all-in-one machine update content to execute the smart computer TV all-in-one machine control method.

[0010] The present invention collects multi-view images of the integrated machine, and the generated three-dimensional scene grid can provide rich spatial information and support a comprehensive understanding of complex environments. The scene element analysis further extracts key environmental features, and the formed scene element set provides a basis for subsequent interaction design, ensuring that the system has sufficient environmental context information when processing user requests, improving the intelligent level of interaction, and tracking the user's limbs. The generated user limb image set can capture the user's movements and postures in real time, enhancing the system's ability to understand user behavior. The use environment tone generated by environmental simulation and restoration provides a real interaction background for the system, ensuring that the system can flexibly respond to user body language and environmental changes. After the acquired integrated machine display content is subjected to content semantic analysis, the generated content semantic data provides support for understanding the core meaning of the displayed content. The interaction intensity calculation is combined with the use environment basis The tone can quantify the degree of interaction between users and content. The generated content interaction intensity data provides a reliable basis for subsequent user intention prediction. The user body language is inferred based on the tone of the usage environment. The formed user body language provides the system with in-depth insights into user intentions. The user intention prediction combined with the content interaction intensity data can accurately capture the user's operation needs. The generated predicted user operation intention provides precise guidance for the system to adjust the displayed content in real time. By adjusting the content of the predicted user operation intention, the generated simulated adjustment content can better adapt to the user's current needs. The composition optimization of the scene element set ensures that the visual effect of the content display is consistent with user expectations. The final all-in-one update content realizes the efficient control of the smart computer TV all-in-one, improves the intelligence and personalization of the overall interactive experience, and promotes the intelligent development of the home entertainment system.

[0011] Preferably, step S1 comprises the following steps:

[0012] Step S11: scanning and collecting the surrounding environment of the integrated machine at multiple angles to obtain a multi-view image of the integrated machine; performing multi-view image registration on the multi-view image of the integrated machine to obtain a registered image data set;

[0013] Step S12: performing point cloud conversion on the registered image data set to obtain a scene point cloud data set; performing grid division on the scene point cloud data set to generate an initial grid data set;

[0014] Step S13: Optimizing the topological structure of the initial mesh data set to generate a three-dimensional scene mesh;

[0015] Step S14: performing scene region segmentation on the three-dimensional scene grid to obtain a region labeling data set; performing spatial relationship analysis on the region labeling data set to generate a scene spatial relationship data set;

[0016] Step S15: extracting scene element attributes from the three-dimensional scene grid to obtain attribute feature data; constructing a hierarchical relationship for the attribute feature data based on the scene spatial relationship data set to generate a scene element set.

[0017] The present invention performs multi-angle scanning and acquisition of the environment around the all-in-one machine, and the generated multi-view images lay the foundation for a comprehensive understanding of the environment. The multi-view image registration ensures that images under different viewpoints can be accurately aligned to form a registered image data set. The registered image data set is converted into a point cloud, and the generated scene point cloud data set can provide rich three-dimensional spatial information. The grid division process converts the point cloud data into an initial grid data set, which provides a structured data basis for subsequent three-dimensional modeling and analysis. The initial grid data set is optimized through topological structure, and the generated three-dimensional scene grid has higher geometric accuracy and topological consistency, which effectively improves the visualization effect and analysis ability of the scene. The three-dimensional scene grid is segmented into scene areas, and the obtained area marking data set provides a basis for the identification and management of different environmental areas. The scene space relationship data set generated by the spatial relationship analysis can describe the relationship and interaction between each area. By extracting the scene element attributes, the generated attribute feature data provides detailed data support for the feature description of each element in the environment. The hierarchical relationship of the attribute feature data is constructed based on the scene space relationship data set, and the generated scene element set provides rich background information for subsequent content adjustment and user interaction, which improves the intelligence level of the system and the user experience.

[0018] Preferably, step S2 comprises the following steps:

[0019] Step S21: Detect skeleton key points on the multi-view image of the integrated device to obtain two-dimensional key point coordinate data; perform spatial projection conversion on the two-dimensional key point coordinate data to generate three-dimensional key point data;

[0020] Step S22: performing time series correlation analysis on the three-dimensional key point data to obtain key point trajectory data; performing posture reconstruction on the key point trajectory data to generate a user limb image set;

[0021] Step S23: performing spatial position calculation on the scene element set to obtain element position data; performing spatial topology reconstruction on the scene element set based on the element position data to generate scene topology data;

[0022] Step S24: performing element restoration mapping on the scene topology data based on the scene element set to obtain the usage environment tone.

[0023] The present invention performs skeleton key point detection on the multi-view images of the all-in-one machine, and the generated two-dimensional key point coordinate data provides a basis for the user's posture analysis. The spatial projection conversion converts the two-dimensional data into three-dimensional key point data, thereby enhancing the system's three-dimensional perception ability of the user's actions. The three-dimensional key point data is subjected to time-series correlation analysis, and the generated key point trajectory data can capture the dynamic changes of the user's actions. The posture reconstruction further forms a user limb image set, providing a richer and more intuitive user behavior expression. The spatial position of the scene element set is calculated, and the obtained element position data provides support for the precise positioning of each element in the environment. The spatial topology reconstruction integrates the element position data into scene topology data, ensuring that the system can effectively understand the environmental structure and element relationship. The scene topology data is restored and mapped based on the scene element set. The generated usage environment tone provides the system with a real environmental background, ensuring that the content display during user interaction matches the actual usage scenario, improving the naturalness and comfort of the user experience, and enhancing the overall intelligent level of interaction.

[0024] Preferably, step S3 comprises the following steps:

[0025] Step S31: acquiring the display content of the integrated machine; performing text recognition and extraction on the display content of the integrated machine to obtain text information data; performing word segmentation processing on the text information data to generate a word-element data set;

[0026] Step S32: performing part-of-speech tagging analysis on the word-meta data set to obtain part-of-speech tagging data; performing syntactic dependency analysis on the part-of-speech tagging data to obtain grammatical structure data;

[0027] Step S33: Calculate semantic vectors of the word-metadata set based on the grammatical structure data to generate content semantic data;

[0028] Step S34: extracting the theme features of the content semantic data to obtain theme feature data; performing scene feature mapping on the usage environment tone according to the theme feature data to generate scene mapping parameters;

[0029] Step S35: Calculate the interactive parameters of the scene mapping parameters to obtain scene interactive data; perform correlation analysis on the subject feature data and the scene interactive data to generate correlation feature data;

[0030] Step S36: Perform comprehensive calculation of the interaction strength of the associated feature data to generate content interaction strength data.

[0031] The present invention obtains the display content of the integrated machine and performs text recognition and extraction, and the generated text information data provides a basis for content analysis. The word segmentation process subdivides the text content into word-meta data sets, thereby enhancing the understanding of the content structure. The word-meta data sets are subjected to part-of-speech tagging and parsing, and the obtained part-of-speech tagging data lays a foundation for the semantic analysis of the content. The grammatical structure data generated by the syntactic dependency method analysis can reveal the relationship between words. The content semantic data generated by the semantic vector calculation based on the grammatical structure data provides a rich information basis for the subsequent theme feature extraction. The extraction of the theme feature data provides a strong support for the scene feature mapping of the use environment tone. The generated scene mapping parameters can realize the effective association between the content and the environment. The scene interaction data obtained by calculating the scene mapping parameters through the interaction parameters can quantify the degree of interaction between the user and the content. The correlation feature data generated by the correlation analysis of the theme feature data and the scene interaction data provides a basis for personalized recommendation and content adjustment. The interaction strength of the correlation feature data is comprehensively calculated, and the generated content interaction strength data provides a specific standard for the system to optimize the user experience, ensure that the content display can adapt to the user's needs and environmental changes in real time, and improve the intelligence level of the overall interactive experience.

[0032] Preferably, step S4 comprises the following steps:

[0033] Step S41: positioning joint points of the user's limb image set to obtain joint coordinate data; calculating posture angles of the joint coordinate data to obtain posture angle data;

[0034] Step S42: performing motion trajectory analysis on the posture angle data to obtain motion trajectory data; performing action pattern recognition on the motion trajectory data to obtain action pattern data;

[0035] Step S43: performing an environment association analysis on the action mode data based on the usage environment keynote to obtain associated behavior feature data; performing body language inference on the associated behavior feature data to generate user body language;

[0036] Step S44: Perform interactive analysis on the user's body language based on the content interaction intensity data to obtain interactive feature data; perform user intention prediction on the interactive feature data to generate predicted user operation intention.

[0037] The present invention locates the joint points of the user's limb image set, and the generated joint coordinate data provides a basis for user posture analysis. The posture angle data further obtained by posture angle calculation can describe the user's body posture in detail. The posture angle data is subjected to motion trajectory analysis, and the generated motion trajectory data can capture the dynamic changes of the user. The action pattern data further formed by action pattern recognition provides support for identifying the user's specific behavior. Based on the usage environment tone, the action pattern data is subjected to environmental association analysis, and the obtained associated behavior feature data can reveal the relationship between the user behavior and the environment, and provide background information for body language speculation. By performing body language speculation on the associated behavior feature data, the generated user body language provides important clues for the system to understand the user's intention, ensuring that the interaction process is more natural and intuitive. The user's body language is interactively analyzed according to the content interaction intensity data, and the generated interaction feature data can quantify the degree of interaction between the user and the system, providing an in-depth understanding of the user's behavior. The predicted user operation intention generated by the interaction feature data through user intention prediction provides a basis for the system to achieve personalized and intelligent feedback, ensuring that the system can adjust the content and interaction method in time when user needs change.

[0038] Preferably, step S43 includes the following steps:

[0039] Defining the spatial scope of the use environment keynote to obtain spatial scope data; performing environmental restriction analysis on the spatial scope data based on the use environment keynote to generate environmental restriction data;

[0040] Extracting scene rules from environmental restriction data to obtain scene rule data; mapping constraint conditions from the scene rule data to generate scene constraint data;

[0041] Extracting action features from the action pattern data to obtain initial action feature data; performing constraint mapping on the initial action feature data based on the scene constraint data to obtain constraint adjustment data;

[0042] Optimizing and adjusting the initial behavior characteristic data based on the constraint adjustment data to generate behavior characteristic data;

[0043] Perform body language analysis on the behavior feature data to obtain action analysis data; perform semantic mapping processing on the action analysis data to generate semantic mapping data;

[0044] The expression pattern is extracted from the semantic mapping data to obtain the expression pattern data; the expression language of the semantic mapping data is inferred based on the expression pattern data to generate the user's body language.

[0045] The present invention defines the spatial scope of the usage environment tone, and the generated spatial scope data provides a clear environmental boundary for subsequent analysis, ensuring that the system can effectively identify and respond to behaviors within the specified range. Environmental restriction analysis is performed based on spatial scope data, and the generated environmental restriction data can identify behavioral constraints in a specific environment, providing a basis for subsequent scene rule extraction. The extraction of scene rule data provides the system with norms and restrictions on user behavior, ensuring that users can follow specific behavior patterns during the interaction process and reduce unnecessary misoperations. Constraint condition mapping is performed on scene rule data, and the generated scene constraint data can effectively combine the environment with user behavior, ensuring that the system can take the actual environment into account when executing user instructions. The action pattern data is subjected to action feature extraction, and the generated initial behavior feature data provides a basis for the analysis of user actions. The detailed information is obtained, and the initial behavior feature data is constrained and mapped based on the scene constraint data. The generated constraint adjustment data can optimize the user behavior recognition process. The behavior feature data is optimized and adjusted, and the final behavior feature data generated provides the system with accurate user action representation. The body language analysis processes the behavior feature data, and the generated action analysis data provides in-depth analysis for user action understanding. The action analysis data is processed by semantic mapping, and the generated semantic mapping data can associate the user's specific actions with the meaning behind them. The expression pattern extraction analyzes the semantic mapping data, and the generated expression pattern data provides a basis for the system to identify the user's expression method. The expression language is inferred based on the expression pattern data, and the generated user body language provides important clues for the system to understand the user's emotions and intentions, thereby improving the naturalness and accuracy of the entire interactive experience.

[0046] Preferably, step S5 comprises the following steps:

[0047] Step S51: performing operation content mapping on the predicted user operation intention to obtain simulated operation data; simulating content change of the all-in-one machine display content based on the predicted user operation intention based on the simulated operation data to obtain simulated adjustment content;

[0048] Step S52: Calculate the user visual path for the scene element set to obtain visual path data;

[0049] Step S53: performing picture composition analysis on the simulated adjustment content based on the visual path data to obtain composition data; performing display effect enhancement processing on the composition data to obtain effect enhancement data;

[0050] Step S54: optimizing the composition of the simulated adjustment content according to the effect enhancement data, generating all-in-one machine update content, and executing the intelligent computer TV all-in-one machine control method.

[0051] The present invention maps the operation content to the predicted user operation intention, and the generated simulated operation data provides the system with a specific response to the user's needs, ensuring that the system can adapt to the user's intention in real time during user interaction, and simulates the content change of the integrated machine display content based on the simulated operation data, and the generated simulated adjustment content can intuitively reflect the user's potential needs, and the scene element set is used to calculate the user's visual path, and the obtained visual path data provides insights into the user's attention distribution, which helps the system understand where the user is paying attention, and the simulated adjustment content is analyzed based on the visual path data. The generated composition data can reveal which elements can best attract the user's attention. The composition data is processed by display effect enhancement, and the generated effect enhancement data can enhance the visual appeal and improve the user's viewing experience, and the composition of the simulated adjustment content is optimized according to the effect enhancement data. The generated integrated machine update content ensures that the information is presented in the best visual way, realizes dynamic adjustment and optimization, and ensures that the user can obtain personalized and efficient content display in different environments and situations, thereby enhancing the control ability and user stickiness of the smart computer TV integrated machine.

[0052] Preferably, step S53 includes the following steps:

[0053] Performing gaze point distribution analysis on the visual path data to obtain gaze distribution data; performing composition grid division on the gaze distribution data to obtain gaze grid layout data;

[0054] Performing golden section calculation on the gaze grid layout data to generate segmentation ratio data; optimizing the picture composition of the simulated adjustment content based on the segmentation ratio data to obtain composition data;

[0055] Performing perspective switching analysis on the composition data to obtain a perspective data set; performing viewing perspective simulation on the perspective data set to obtain a simulated viewing data set;

[0056] Perform comfort calculation on the simulated viewing data set to generate comfort data; perform optimized viewing angle analysis on the viewing angle data set based on the comfort data to generate optimized viewing angle data;

[0057] The composition data is processed for display effect enhancement according to the optimized viewing angle data to obtain effect enhancement data.

[0058] The present invention performs gaze point distribution analysis on visual path data, and the generated gaze distribution data provides a basis for identifying the user's focus point, ensuring that the system can understand the user's visual preference. The gaze distribution data is divided into a composition grid, and the obtained gaze grid layout data can clarify the distribution of the user's attention on the screen. The golden section calculation is performed on the gaze grid layout data, and the generated segmentation ratio data provides aesthetic guidance for the picture composition, ensuring that the content display conforms to the visual balance and harmony. The picture composition of the simulated adjustment content is optimized based on the segmentation ratio data, and the obtained composition data can effectively improve the readability and attractiveness of the information. After the perspective switching analysis, the generated perspective data set provides a basis for a diversified viewing experience, ensuring that the system can change according to different perspectives. Adjust the content display and simulate the viewing perspective of the perspective data set. The obtained simulated viewing data set provides intuitive feedback for the user's experience under different perspectives. The simulated viewing data set is subjected to comfort calculation. The generated comfort data can quantify the comfort level of the user's viewing experience. The perspective data set is optimized for perspective analysis based on the comfort data. The generated optimized perspective data ensures that the system can recommend the best viewing angle. The composition data is enhanced for display effect according to the optimized perspective data. The obtained effect enhancement data can further enhance the visual effect of the content, realize comprehensive optimization of the user's visual experience, ensure that the information is not only attractive, but also can provide high-quality interactive experience under diverse perspectives and comfort requirements, and enhance the user stickiness and satisfaction of smart computer-TV all-in-ones.

[0059] The present invention also provides a computer-TV all-in-one control system for executing the above-mentioned computer-TV all-in-one control method, and the computer-TV all-in-one control system comprises:

[0060] The image acquisition module is used to acquire multi-view images of the integrated machine; perform spatial grid modeling on the multi-view images of the integrated machine to obtain a three-dimensional scene grid; perform scene element analysis on the three-dimensional scene grid to generate a scene element set;

[0061] The environment analysis module is used to track the user's body in the multi-view images of the all-in-one machine to generate a set of user's body images; perform environmental simulation and restoration on the scene element set to obtain the tone of the use environment;

[0062] The content deconstruction module is used to obtain the content displayed on the all-in-one machine; perform content semantic analysis on the content displayed on the all-in-one machine to obtain content semantic data; calculate the interaction intensity of the content semantic data according to the tone of the usage environment to generate content interaction intensity data;

[0063] The operation recognition module is used to infer the body language of the user's body image set based on the tone of the use environment to obtain the user's body language; predict the user's intention based on the user's body language according to the content interaction intensity data, and generate the predicted user operation intention;

[0064] The control adjustment module is used to simulate the content adjustment of the all-in-one display content based on the predicted user operation intention to obtain the simulated adjustment content; optimize the composition of the simulated adjustment content according to the scene element set to generate the all-in-one update content to execute the smart computer TV all-in-one control method.

[0065] The invention provides comprehensive environmental information through the acquisition and processing of multi-view images, ensuring that the system can accurately construct a three-dimensional scene grid. The generation of the three-dimensional scene grid and the analysis of scene elements provide structured data support for subsequent interactions, ensuring that the system can effectively identify and manage various elements in a complex environment. The environmental analysis module can track the user's body movements in real time, and the generated user body image set provides a basis for understanding user behavior. The generation of the use environment tone is restored through environmental simulation, ensuring the adaptability of the system in different environments, and improving the consistency and naturalness of user experience. The content deconstruction module realizes the understanding of the deep meaning of the content through semantic analysis of the displayed content. The interaction intensity calculation is based on the environmental tone, ensuring that the system can quantify the interaction between the user and the content. The operation recognition module can accurately capture the user's needs and emotions through body language speculation and user intention prediction, providing a basis for the system to adjust the content in real time. The control adjustment module ensures that the content display is more in line with the user's visual expectations through simulation adjustment of content and composition optimization, enhancing the effectiveness and attractiveness of information transmission, realizing dynamic understanding and response to user behavior, ensuring that personalized and intelligent experience is provided in a variety of usage scenarios, and enhancing the user stickiness and satisfaction of the smart computer TV all-in-one machine.

[0066] The present invention also provides a computer-TV all-in-one control device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, for executing any one of the computer-TV all-in-one control methods described above.

[0067] The present invention supports complex data processing through efficient computing power, ensuring that the system can execute multiple control tasks in real time, improving the response speed and efficiency of operations, and the memory provides a large-capacity storage space, allowing the system to store multi-view images, user body data and content semantic information, ensuring rapid access and processing of data, and realizing the complete execution of the computer-TV all-in-one control method, ensuring that the functional modules of the system can work together, and improving the overall system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A schematic diagram of the steps of a method for controlling a computer-TV all-in-one machine;

[0069] Figure 2 for Figure 1Detailed implementation steps of step S2 in the flowchart;

[0070] Figure 3 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0071] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0072] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0073] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0074] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0075] To achieve this, please refer to Figures 1 to 3 , a computer-TV all-in-one control method, comprising the following steps:

[0076] Step S1: collecting multi-view images of the integrated machine; performing spatial grid modeling on the multi-view images of the integrated machine to obtain a three-dimensional scene grid; performing scene element analysis on the three-dimensional scene grid to generate a scene element set;

[0077] Step S2: tracking the user's limbs on the multi-view images of the all-in-one device to generate a user limb image set; performing environmental simulation restoration on the scene element set to obtain the use environment tone;

[0078] Step S3: acquiring the display content of the all-in-one machine; performing content semantic analysis on the display content of the all-in-one machine to obtain content semantic data; calculating the interaction intensity of the content semantic data according to the tone of the usage environment to generate content interaction intensity data;

[0079] Step S4: inferring the body language of the user's body image set based on the tone of the usage environment to obtain the user's body language; predicting the user's intention based on the user's body language according to the content interaction intensity data to generate a predicted user operation intention;

[0080] Step S5: adjusting the display content of the all-in-one machine based on the predicted user operation intention to obtain simulated adjustment content; optimizing the composition of the simulated adjustment content according to the scene element set to generate all-in-one machine update content to execute the smart computer TV all-in-one machine control method.

[0081] The present invention collects multi-view images of the integrated machine, and the generated three-dimensional scene grid can provide rich spatial information and support a comprehensive understanding of complex environments. The scene element analysis further extracts key environmental features, and the formed scene element set provides a basis for subsequent interaction design, ensuring that the system has sufficient environmental context information when processing user requests, improving the intelligent level of interaction, and tracking the user's limbs. The generated user limb image set can capture the user's movements and postures in real time, enhancing the system's ability to understand user behavior. The use environment tone generated by environmental simulation and restoration provides a real interaction background for the system, ensuring that the system can flexibly respond to user body language and environmental changes. After the acquired integrated machine display content is subjected to content semantic analysis, the generated content semantic data provides support for understanding the core meaning of the displayed content. The interaction intensity calculation is combined with the use environment basis The tone can quantify the degree of interaction between users and content. The generated content interaction intensity data provides a reliable basis for subsequent user intention prediction. The user body language is inferred based on the tone of the usage environment. The formed user body language provides the system with in-depth insights into user intentions. The user intention prediction combined with the content interaction intensity data can accurately capture the user's operation needs. The generated predicted user operation intention provides precise guidance for the system to adjust the displayed content in real time. By adjusting the content of the predicted user operation intention, the generated simulated adjustment content can better adapt to the user's current needs. The composition optimization of the scene element set ensures that the visual effect of the content display is consistent with user expectations. The final all-in-one update content realizes the efficient control of the smart computer TV all-in-one, improves the intelligence and personalization of the overall interactive experience, and promotes the intelligent development of the home entertainment system.

[0082] In the embodiment of the present invention, refer to Figure 1, is a schematic flow chart of the steps of a computer-TV all-in-one control method of the present invention. In this example, the computer-TV all-in-one control method includes the following steps:

[0083] Step S1: collecting multi-view images of the integrated machine; performing spatial grid modeling on the multi-view images of the integrated machine to obtain a three-dimensional scene grid; performing scene element analysis on the three-dimensional scene grid to generate a scene element set;

[0084] In this embodiment, when collecting multi-view images of the all-in-one machine, a plurality of fixed-angle camera devices are set up. These camera devices are arranged at multiple angles such as the left, right, top, and front of the all-in-one machine to collect the external environment image information of the all-in-one machine in all directions. After obtaining the multi-view images, the image data is converted into a three-dimensional stereo model through a spatial grid modeling method, and a point cloud generation algorithm is used to locate each pixel in the image in a three-dimensional coordinate axis to construct a complete three-dimensional scene grid. Afterwards, each element in the three-dimensional scene grid is extracted and classified using image segmentation and identification technology, and classified into different scene elements, such as furniture, walls, windows, etc. Through such scene element analysis, a scene element set with a specific classification is generated to provide basic data support for subsequent operations.

[0085] Step S2: tracking the user's limbs on the multi-view images of the all-in-one device to generate a user limb image set; performing environmental simulation restoration on the scene element set to obtain the use environment tone;

[0086] In this embodiment, multi-view images are input into the human posture recognition system, user limb tracking is performed based on a deep learning model, and OpenPose (an open source posture estimation framework) is used to identify the user's joint position and limb movement trajectory, thereby generating a user limb image set. These image sets contain the user's action status in different scenes, such as gesture direction, sitting posture, standing posture, etc. After the user limb image set is generated, it is combined with the scene element set generated in the previous step, based on the three-dimensional environment reconstruction algorithm, to restore the visual appearance and spatial structure of the environment. In the simulation restoration process, a light and shadow adjustment algorithm is used to create a simulation model based on the lighting, color and other information of the real environment, and generate the final usage environment tone, providing data support for the environmental adaptability of the all-in-one machine.

[0087] Step S3: acquiring the display content of the all-in-one machine; performing content semantic analysis on the display content of the all-in-one machine to obtain content semantic data; calculating the interaction intensity of the content semantic data according to the tone of the usage environment to generate content interaction intensity data;

[0088] In this embodiment, the content data currently displayed on the all-in-one machine is captured through the content acquisition module, and the captured content data is subjected to content semantic analysis through the image parsing model of computer vision to parse the meaning of elements such as images and texts. For example, natural language processing (NLP) and visual detection technology are used to analyze the semantic labels of each information in the displayed content. After the content semantic data is obtained, the interaction intensity of the content semantic data is calculated based on the tone of the usage environment. A weighted algorithm is used to quantitatively evaluate the interactive relationship between the content and the usage environment based on the emotional tone of the displayed content, the interaction needs, and other information to generate content interaction intensity data.

[0089] Step S4: inferring the body language of the user's body image set based on the tone of the usage environment to obtain the user's body language; predicting the user's intention based on the user's body language according to the content interaction intensity data to generate a predicted user operation intention;

[0090] In this embodiment, based on the tone of the usage environment, a body language recognition model is used to identify and infer the user's body image set, and features of the body image set are extracted through a convolutional neural network (CNN). The user's current body language is analyzed and body language feature data is output. For example, specific actions such as waving, clicking, and stretching can be identified. Subsequently, based on the content interaction intensity data and the correlation between content characteristics and user posture, the user's intention is predicted for body language. The KNN (K nearest neighbor algorithm) is used to classify and predict the user's intention, and the user's operation motivation, such as pausing, browsing, etc., is identified, thereby generating a prediction of the user's operation intention.

[0091] Step S5: adjusting the display content of the all-in-one machine based on the predicted user operation intention to obtain simulated adjustment content; optimizing the composition of the simulated adjustment content according to the scene element set to generate all-in-one machine update content to execute the smart computer TV all-in-one machine control method.

[0092] In this embodiment, the current displayed content is adjusted according to the user operation intention prediction data, and the layout, interaction mode, etc. of the content are adjusted according to the predicted user behavior. The adjusted display content is simulated through an image generation model based on a convolutional neural network (CNN) and a generative adversarial network (GAN), and an optimized display interface is output, such as highlighting important visual information or adjusting the layout ratio of the page. After the simulated adjustment content is generated, the content composition is optimized using a scene element set, and the displayed content is aligned with key elements in the actual scene, such as screen distance, angle, light source position, etc., to generate updated content for the all-in-one machine for real-time display.

[0093] Preferably, step S1 comprises the following steps:

[0094] Step S11: scanning and collecting the surrounding environment of the integrated machine at multiple angles to obtain a multi-view image of the integrated machine; performing multi-view image registration on the multi-view image of the integrated machine to obtain a registered image data set;

[0095] Step S12: performing point cloud conversion on the registered image data set to obtain a scene point cloud data set; performing grid division on the scene point cloud data set to generate an initial grid data set;

[0096] Step S13: Optimizing the topological structure of the initial mesh data set to generate a three-dimensional scene mesh;

[0097] Step S14: performing scene region segmentation on the three-dimensional scene grid to obtain a region labeling data set; performing spatial relationship analysis on the region labeling data set to generate a scene spatial relationship data set;

[0098] Step S15: extracting scene element attributes from the three-dimensional scene grid to obtain attribute feature data; constructing a hierarchical relationship for the attribute feature data based on the scene spatial relationship data set to generate a scene element set.

[0099] In this embodiment, the scanning and acquisition of the surrounding environment is started through the multi-angle camera module of the all-in-one machine, and the camera equipment at different angles is adjusted, such as the top, left and right sides, and the front, so as to obtain the environmental image data of the all-in-one machine at multiple angles, and the image data is transmitted to the multi-view image processing module. The image registration algorithm based on feature points is used to perform registration operations on the image data collected at each angle. During the registration process, the feature extraction method (such as SIFT, i.e., scale-invariant feature transform) is used to extract the feature points of each image to identify the same spatial features, and the multi-view images are matched and aligned. After all the view images are registered, a complete multi-view registration image data set of the all-in-one machine is generated. , use the 3D reconstruction module to convert the registered image data set into a point cloud, convert the pixel points in each registered image into points in 3D space, and form a point cloud data set of the 3D scene. In the point cloud conversion process, the structured light scanning method or deep learning point cloud generation algorithm is used to convert the spatial coordinate information of the multi-view image into a fine point cloud model. After the point cloud data is generated, the spatial grid division technology is applied to divide the point cloud data set into grid units of uniform size to obtain the initial grid data set. The initial grid is organized and segmented using the octree structure so that the grid data presents a clear hierarchical structure. The initial grid data set is input into the topology optimization module , the mesh data set is optimized through the topological structure optimization algorithm, and the surface reconstruction method and Delaunay triangulation algorithm are used to clear and reconstruct the redundant facets in the mesh to ensure the uniformity and continuity of the mesh structure. In particular, the Laplace smoothing algorithm is used to smooth the mesh parts with noise points or cracks to eliminate irregular protrusions or depressions, and finally form a complete 3D scene mesh. In the scene segmentation module, the 3D scene mesh is segmented according to the topological structure of the 3D scene mesh and the geometric characteristics of the object, and the mesh data is marked with regions based on the region growth algorithm, such as marking each segmented region with different colors or numbers. After the regions are marked, a spatial relationship analysis algorithm is used to establish a spatial relationship model between regions according to the spatial orientation, adjacency, etc. of each region, and a scene spatial relationship dataset is generated. The scene spatial relationship dataset and the three-dimensional scene grid data are imported into the feature extraction module, and the scene element attributes are extracted from each region of the three-dimensional scene grid. The shape, material, texture and other attribute features of the objects in each region are extracted using a deep learning feature segmentation algorithm to generate preliminary attribute feature data. The attribute feature data is combined with the scene spatial relationship dataset, and a hierarchical relationship between scene elements is constructed through hierarchical clustering. In the hierarchical relationship, elements with similar spatial positions or attribute features are classified and combined to finally generate a scene element set.The scene element set contains the attribute characteristics of each scene element and its hierarchical relationship, which is used for subsequent operations.

[0100] Preferably, step S2 comprises the following steps:

[0101] Step S21: Detect skeleton key points on the multi-view image of the integrated device to obtain two-dimensional key point coordinate data; perform spatial projection conversion on the two-dimensional key point coordinate data to generate three-dimensional key point data;

[0102] Step S22: performing time series correlation analysis on the three-dimensional key point data to obtain key point trajectory data; performing posture reconstruction on the key point trajectory data to generate a user limb image set;

[0103] Step S23: performing spatial position calculation on the scene element set to obtain element position data; performing spatial topology reconstruction on the scene element set based on the element position data to generate scene topology data;

[0104] Step S24: performing element restoration mapping on the scene topology data based on the scene element set to obtain the usage environment tone.

[0105] As an example of the present invention, see Figure 2 In this example, step S2 includes:

[0106] Step S21: Detect skeleton key points on the multi-view image of the integrated device to obtain two-dimensional key point coordinate data; perform spatial projection conversion on the two-dimensional key point coordinate data to generate three-dimensional key point data;

[0107] In this embodiment, the multi-view image data acquisition module of the all-in-one machine is used to perform skeletal key point detection on the collected image data, and a posture estimation algorithm based on deep learning, such as OpenPose or MediaPipe skeletal detection algorithm, is used to detect the user's main skeletal joints from each image, and the skeletal joint point coordinate data is used to generate a two-dimensional key point coordinate data set. Next, the two-dimensional key point coordinate data is spatially projected and transformed through a spatial projection transformation module. By combining the multi-view key point data with an optical geometry calculation model, a projection mapping of two-dimensional plane coordinates to three-dimensional space coordinates is realized, and a three-dimensional key point data set is generated for subsequent analysis of user posture and action behavior.

[0108] Step S22: performing time series correlation analysis on the three-dimensional key point data to obtain key point trajectory data; performing posture reconstruction on the key point trajectory data to generate a user limb image set;

[0109] In this embodiment, the generated three-dimensional key point data is input into the timing analysis module, and the continuous key point data is subjected to timing correlation analysis by introducing a sliding window mechanism. During the analysis process, the long short-term memory network (LSTM) or Kalman filter algorithm is used to track the trajectory changes of the key point coordinates at each moment to obtain the motion trajectory data of the user's skeletal key points. Based on the motion trajectory data, the user's limb image is further reconstructed in the posture reconstruction module. The trajectory data is converted into a skeletal connection model and the user's limb contour is rendered using a three-dimensional skeleton interpolation and posture smoothing algorithm to generate a user limb image set.

[0110] Step S23: performing spatial position calculation on the scene element set to obtain element position data; performing spatial topology reconstruction on the scene element set based on the element position data to generate scene topology data;

[0111] In this embodiment, the scene element set data generated in the previous stage is input into the spatial position calculation module, and the spatial coordinates of each scene element in the scene element set are accurately calculated. The 3D spatial calculation model is applied to analyze the three-dimensional position, size and relative relationship of each element, and generate corresponding element position data. The element position data set is used to assign each element its absolute coordinates in the three-dimensional space. Based on the generated element position data, the spatial topology of each element in the scene element set is reconstructed. Through the topology reconstruction algorithm, all scene elements are connected in the form of nodes and edges, and the topological structure is optimized using a spatial relationship model based on node association, and finally the scene topology data is generated.

[0112] Step S24: performing element restoration mapping on the scene topology data based on the scene element set to obtain the usage environment tone.

[0113] In this embodiment, the scene element set and the scene topology data generated in the previous step are used to restore and map the element positions, attributes and environmental influencing factors in the scene through a mapping module. The interactive relationship between each element in the three-dimensional scene is reproduced through an element restoration algorithm. At the same time, the environmental tone in the scene, such as light intensity, color saturation, background depth, etc., is simulated to generate the environmental tone for use in the all-in-one machine control method.

[0114] Preferably, step S3 comprises the following steps:

[0115] Step S31: acquiring the display content of the integrated machine; performing text recognition and extraction on the display content of the integrated machine to obtain text information data; performing word segmentation processing on the text information data to generate a word-element data set;

[0116] Step S32: performing part-of-speech tagging analysis on the word-meta data set to obtain part-of-speech tagging data; performing syntactic dependency analysis on the part-of-speech tagging data to obtain grammatical structure data;

[0117] Step S33: Calculate semantic vectors of the word-metadata set based on the grammatical structure data to generate content semantic data;

[0118] Step S34: extracting the theme features of the content semantic data to obtain theme feature data; performing scene feature mapping on the usage environment tone according to the theme feature data to generate scene mapping parameters;

[0119] Step S35: Calculate the interactive parameters of the scene mapping parameters to obtain scene interactive data; perform correlation analysis on the subject feature data and the scene interactive data to generate correlation feature data;

[0120] Step S36: Perform comprehensive calculation of the interaction strength of the associated feature data to generate content interaction strength data.

[0121] As an example of the present invention, see Figure 3 In this example, step S3 includes:

[0122] Step S31: acquiring the display content of the integrated machine; performing text recognition and extraction on the display content of the integrated machine to obtain text information data; performing word segmentation processing on the text information data to generate a word-element data set;

[0123] In this embodiment, the content currently displayed on the all-in-one display screen is captured by the content acquisition module, and the captured content is input into the text recognition module. The text information in the displayed content is scanned line by line using the OCR (Optical Character Recognition) technology, and the text in the image is converted into text information data. Subsequently, the text information data is segmented word by word by the word segmentation processing module, and the maximum forward matching algorithm is applied to convert the continuous text data into independent word units, and a word unit data set is generated for subsequent processing.

[0124] Step S32: performing part-of-speech tagging analysis on the word-meta data set to obtain part-of-speech tagging data; performing syntactic dependency analysis on the part-of-speech tagging data to obtain grammatical structure data;

[0125] In this embodiment, each word in the word data set is tagged with a part-of-speech tag by a natural language processing module, and the part-of-speech role (such as noun, verb, adjective, etc.) of each word in the sentence is identified by using a part-of-speech tagging algorithm based on a hidden Markov model, and part-of-speech tagging data containing the part-of-speech label of each word is generated. Next, the part-of-speech tagging data is input into a syntactic analysis module, and a syntactic dependency analysis algorithm (such as the dependency parsing toolkit of Stanford NLP) is used to parse the syntactic dependency relationship of the sentence structure, so as to obtain the grammatical dependency relationship between each word, and generate grammatical structure data.

[0126] Step S33: Calculate semantic vectors of the word-metadata set based on the grammatical structure data to generate content semantic data;

[0127] In this embodiment, the generated grammatical structure data and word-meta data set are input into a semantic vector calculation module, and the semantic vector representation of each word-meta is calculated through a word embedding model (such as Word2Vec or BERT). The grammatical structure information is used as a weight to adjust the vector calculation to highlight important words in the syntactic hierarchy, forming comprehensive content semantic data, thereby providing accurate semantic expression for subsequent content topic extraction.

[0128] Step S34: extracting the theme features of the content semantic data to obtain theme feature data; performing scene feature mapping on the usage environment tone according to the theme feature data to generate scene mapping parameters;

[0129] In this embodiment, the content semantic data is input into the topic feature extraction module, and the topic information in the content is extracted using an algorithm based on the LDA (Latent Dirichlet Allocation) topic model. The semantic data is clustered into specific topics through multiple iterations to finally obtain the topic feature data. Next, the topic feature data is mapped to the generated usage environment tone through scene feature mapping, and the spatial feature mapping algorithm is applied to map different topics to the corresponding scene environment features to generate scene mapping parameters.

[0130] Step S35: Calculate the interactive parameters of the scene mapping parameters to obtain scene interactive data; perform correlation analysis on the subject feature data and the scene interactive data to generate correlation feature data;

[0131] In this embodiment, the scene mapping parameters are input into the interaction parameter calculation module, and the scene feature data and the environmental tone are analyzed for interaction intensity through context analysis and semantic interaction models to obtain the scene interaction data. At the same time, the subject feature data and the scene interaction data are used as input for correlation analysis, and a semantic correlation analysis model based on a convolutional neural network is applied to calculate the correlation between the subject features and the scene interaction in the context to generate correlation feature data.

[0132] Step S36: Perform comprehensive calculation of the interaction strength of the associated feature data to generate content interaction strength data.

[0133] In this embodiment, the generated correlation feature data is input into the interaction intensity calculation module, and each correlation feature is hierarchically scored through the hierarchical analysis method (AHP), and then the scoring results are normalized to calculate the interaction intensity of different theme contents under the current environmental tone, and finally generate content interaction intensity data.

[0134] Preferably, step S4 comprises the following steps:

[0135] Step S41: positioning joint points of the user's limb image set to obtain joint coordinate data; calculating posture angles of the joint coordinate data to obtain posture angle data;

[0136] Step S42: performing motion trajectory analysis on the posture angle data to obtain motion trajectory data; performing action pattern recognition on the motion trajectory data to obtain action pattern data;

[0137] Step S43: performing an environment association analysis on the action mode data based on the usage environment keynote to obtain associated behavior feature data; performing body language inference on the associated behavior feature data to generate user body language;

[0138] Step S44: Perform interactive analysis on the user's body language based on the content interaction intensity data to obtain interactive feature data; perform user intention prediction on the interactive feature data to generate predicted user operation intention.

[0139] In this embodiment, a deep learning joint point detection model is used to identify the joint point positions of the user's limb image set, and the OpenPose or HRNet model is used to identify and locate each joint point of the user's limb (including key points such as shoulder, elbow, wrist, hip, knee, ankle, etc.) frame by frame, and the pixel coordinate data of each joint point is stored as joint coordinate data. After the joint point detection is completed, the posture angle is calculated based on the coordinate information, and the posture angle data between each joint point is calculated by the vector angle formula. For example, for the elbow joint angle, two vectors are formed by the coordinates of the shoulder, elbow, and wrist, and the angle is obtained by the inverse cosine function. The data is stored as posture angle data, and the motion trajectory of the posture angle data is analyzed. The time series of the posture angle data is smoothed using the Kalman filter algorithm to filter out the existing noise and enhance the continuity of the trajectory. Subsequently, the trajectory tracking algorithm (such as dynamic time warping DTW) is used to associate and match the continuous frame data of the posture angle change to generate motion trajectory data. The motion trajectory data is then input into the action pattern recognition module, and the long short-term memory network LSTM is used to analyze the temporal features in the motion trajectory, identify the action mode, and judge the user's action as a specific mode such as stillness, waving, pointing, etc., and generate an action pattern. Based on the generated usage environment tone, the action pattern data is subjected to environment association analysis. The degree of association between the current action pattern and the environment tone is calculated with the help of a fuzzy matching algorithm. For example, in a calm mood, the waving pattern will be identified as a "greeting" intention, while in a tense mood, it will be identified as a "paying attention" intention. After the association analysis is completed, the obtained associated behavior feature data is input into the body language inference module. The body language expression intention of the user is inferred through a body language analysis model (such as an analysis method based on the HMM hidden Markov model). For example, by identifying the combination of body leaning forward and pointing, it is inferred as "paying attention". ” or “need help”, generate user body language and store it as structured data, perform interactive analysis on user body language based on content interaction intensity data, input user body language data and content interaction intensity data into the association analysis model, combine the interaction characteristics between user actions and content, apply collaborative filtering algorithm to calculate the matching degree of different interaction features, generate interaction feature data, then use machine learning-based intention prediction model (such as intention classifier based on multi-layer perceptron MLP) to perform deep feature extraction on interaction feature data, and generate prediction results of user operation intentions based on historical user behavior data.

[0140] Preferably, step S43 includes the following steps:

[0141] Defining the spatial scope of the use environment keynote to obtain spatial scope data; performing environmental restriction analysis on the spatial scope data based on the use environment keynote to generate environmental restriction data;

[0142] Extracting scene rules from environmental restriction data to obtain scene rule data; mapping constraint conditions from the scene rule data to generate scene constraint data;

[0143] Extracting action features from the action pattern data to obtain initial action feature data; performing constraint mapping on the initial action feature data based on the scene constraint data to obtain constraint adjustment data;

[0144] Optimizing and adjusting the initial behavior characteristic data based on the constraint adjustment data to generate behavior characteristic data;

[0145] Perform body language analysis on the behavior feature data to obtain action analysis data; perform semantic mapping processing on the action analysis data to generate semantic mapping data;

[0146] The expression pattern is extracted from the semantic mapping data to obtain the expression pattern data; the expression language of the semantic mapping data is inferred based on the expression pattern data to generate the user's body language.

[0147] In this embodiment, a camera is used to collect an environmental image of the user's current operating space, and a target detection model based on a convolutional neural network (CNN) (for example, YOLOv5) is used to identify boundary elements in the image. By calculating the spatial position of the boundary element and its distance from the user's position, range description data including the boundary of the spatial range is generated. The spatial range data is verified in combination with the rule information about the operating space in the environmental tone to ensure that the defined spatial range can meet the subsequent environmental analysis requirements. Finally, the spatial range data is generated, and a multi-rule environmental restriction parser is called. The environmental restriction rules in the environmental tone and the spatial range data are input into the parser. The parsing rule content includes operating area restriction, ambient light intensity restriction, etc. The parsed environmental restriction rules are compared with the spatial range. Each data item is matched and calculated, and environmental restriction data describing the current environmental restriction conditions is generated by judging whether the rule requirements are met. Based on the generated environmental restriction data, a rule extraction algorithm based on natural language processing (such as the BERT model) is used to extract the scene constraint information implicit in the rule. The scene behavior constraints and operation conditions involved in the environmental restriction data are decomposed by the rule parser, and the rule items related to the scene are output in a structured form to obtain the scene rule data describing the scene characteristics. The scene rule data contains information such as the behavior operation range, lighting requirements, and position dynamic constraints. The multi-dimensional constraint mapping algorithm is used in combination with the scene rule data to map the spatial constraints and lighting constraints in the rules with the specific The abstract constraint conditions in the rules are converted into actual scene constraint data through matrix mapping. The scene constraint data includes the spatial coordinates of the operable area, the range of operation action limits, and the dynamic constraint values ​​related to the environmental conditions. The acquired action pattern data is input into a feature extraction model based on time series analysis (such as a model based on the long short-term memory network LSTM), and the temporal characteristics contained in the action pattern data are analyzed. The user action feature information is extracted through the model, including time span, speed, repetition frequency, etc. The extracted feature data is stored as initial behavior feature data, and the initial behavior feature data is subjected to constraint mapping processing. By constructing constraint mapping The mapping matrix is ​​used to compare the position restriction and motion amplitude restriction in the scene constraint data with the motion features extracted from the initial behavior feature data. The data deviation in the user behavior feature is corrected through multi-dimensional linear mapping to generate constraint adjustment data. The behavior adjustment module based on the optimization algorithm (such as the particle swarm optimization algorithm PSO) is called. The initial behavior feature data and the constraint adjustment data are input into the optimization module. The optimization module jointly optimizes the two sets of data and adjusts the user's motion amplitude and motion trajectory through iterative calculation to make them more consistent with the scene constraint conditions. The optimized result is output as the behavior feature data, and the behavior feature data is input into the body language parser. The parser is based on the pre-trained body language feature library.The behavioral features are compared with the body language features through the pattern matching algorithm to parse out the basic expression features in the user's body language, such as posture tendency and action intention. After the analysis is completed, action analysis data is generated. The behavioral features in the action analysis data are converted into semantic information using a semantic mapping model (for example, a semantic vector model based on Word2Vec). The semantic vector is generated by calculating the similarity between the action features and the semantic space. The user's action semantic features are further derived in combination with situational factors to generate semantic mapping data. The semantic mapping data includes the semantic expression content of the user's action and its meaning in the scene. The expression pattern analysis module based on the clustering algorithm (for example, K-Mean) is called. s clustering), cluster analysis is performed on the semantic vectors contained in the semantic mapping data, the pattern features reflected in the user's expression are extracted, and the expression pattern data is generated. The expression pattern data describes the user's main expression methods and action language patterns in the current environment. The expression pattern data is combined with the semantic mapping data, and the expression language inference model based on deep learning (such as the Transformer model) is used to make a linguistic inference on the user's expression pattern. The relationship between the action features and the semantic expression is learned by the model, and the user's body language content is inferred and structured into standardized user body language description data, and finally the user body language for system control and user interaction is generated. ,

[0148] Preferably, step S5 comprises the following steps:

[0149] Step S51: performing operation content mapping on the predicted user operation intention to obtain simulated operation data; simulating content change of the all-in-one machine display content based on the predicted user operation intention based on the simulated operation data to obtain simulated adjustment content;

[0150] Step S52: Calculate the user visual path for the scene element set to obtain visual path data;

[0151] Step S53: performing picture composition analysis on the simulated adjustment content based on the visual path data to obtain composition data; performing display effect enhancement processing on the composition data to obtain effect enhancement data;

[0152] Step S54: optimizing the composition of the simulated adjustment content according to the effect enhancement data, generating all-in-one machine update content, and executing the intelligent computer TV all-in-one machine control method.

[0153] In this embodiment, by analyzing the predicted user operation intention, the operation instructions that the user will execute are input into the operation mapping module. The module uses a preset operation instruction-content mapping library to match the instructions with the operation content responded by the all-in-one machine. By querying the content association rules contained in the library, simulated operation data matching the operation intention is generated. The simulated operation data includes the interface interaction content that will be executed, display change information and function call parameters. The content change simulation algorithm module is used to input the simulated operation data into the dynamic display engine. The dynamic display engine simulates the changes in the display interface when the user executes related operations based on the display logic rules. The display content is adjusted and the simulation is generated through the real-time simulation screen generation engine. The simulated screen effect includes the content change area, adjustment parameters and simulated display frame sequence, and the adjusted result is stored as the simulated adjustment content. The element position information, size information and relative layout relationship in the scene element set are extracted. Through the user visual attention model based on image processing (such as a model based on a heat map algorithm), the user's line of sight moving path on the screen is simulated, the user's visual trajectory is analyzed, the key points in the trajectory are recorded and marked, and the user's visual path obtained by analysis is stored as visual path data. The visual path data includes the starting point, end point, trajectory shape and dwell time of the line of sight movement. The visual path data and the simulated adjustment content are jointly analyzed. Through composition analysis The algorithm module (for example, an algorithm based on edge detection and point of interest extraction) calculates and simulates the spatial distribution of key elements in the content to match the visual path, extracts the distribution of visual attraction points in the picture, combines the overlap between the user's visual focus and the distribution of the picture area, analyzes the balance and primary and secondary relationships of the picture, and generates composition data. The composition data includes the weight distribution of the picture area and the visual priority ranking between the elements. The composition data is input into the display enhancement module. Through the deep learning-based image enhancement network (for example, a model based on the generative adversarial network GAN), the display parameters such as brightness, contrast, and saturation are adjusted for the visual key areas in the picture, and the low-weight areas are visually weakened to highlight the key points. The detail expression and clarity of the picture are improved through multi-frame processing, and the generated enhancement results are stored as effect enhancement data. The effect enhancement data includes the optimization adjustment parameters of each area of ​​the picture and the visually enhanced image frames. The effect enhancement data and the simulated adjustment content are input into the composition optimization module, and the position of key elements in the simulated picture is fine-tuned using the picture layout optimization algorithm. During the adjustment process, the continuity of the visual path and the prominence of the main content are prioritized, while the distribution of secondary elements is optimized to maintain the overall picture balance. The final optimized picture data is output as the all-in-one machine update content. The update content includes the optimized interface layout and display parameter configuration, which is used to adjust the display effect and interactive content of the all-in-one machine in real time.

[0154] Preferably, step S53 includes the following steps:

[0155] Performing gaze point distribution analysis on the visual path data to obtain gaze distribution data; performing composition grid division on the gaze distribution data to obtain gaze grid layout data;

[0156] Performing golden section calculation on the gaze grid layout data to generate segmentation ratio data; optimizing the picture composition of the simulated adjustment content based on the segmentation ratio data to obtain composition data;

[0157] Performing perspective switching analysis on the composition data to obtain a perspective data set; performing viewing perspective simulation on the perspective data set to obtain a simulated viewing data set;

[0158] Perform comfort calculation on the simulated viewing data set to generate comfort data; perform optimized viewing angle analysis on the viewing angle data set based on the comfort data to generate optimized viewing angle data;

[0159] The composition data is processed for display effect enhancement according to the optimized viewing angle data to obtain effect enhancement data.

[0160] In this embodiment, the user gaze trajectory in the visual path data is analyzed, and a gaze point distribution algorithm based on cluster analysis is adopted. The user's gaze points are aggregated by a K-means clustering method, and the distribution area of ​​the gaze points on the screen is divided into multiple high-density areas and low-density areas. The density distribution of the gaze points in each area and the proportion of the gaze time are calculated to generate gaze distribution data. The gaze distribution data includes the center coordinates of the gaze point distribution, the gaze frequency statistics and the boundary description information of the gaze area. The area in the gaze distribution data is divided by a grid division algorithm. A dynamic grid generation algorithm is adopted to set the size and number of grid units according to the density of the gaze points. A higher density area uses a finer grid division, and a low density area uses a coarser grid division. By adjusting the distribution and shape of the grid, the entire screen is divided into multiple grid units with a hierarchical relationship, and the gaze frequency and gaze area of ​​each grid unit are marked to generate the gaze grid layout data. The gaze grid layout data includes the spatial coordinates and gaze weight of each grid unit. The position and size of the grid unit in the gaze grid layout data are analyzed, and the golden ratio (about 0.618) Calculate the golden ratio distribution of each grid unit relative to the entire screen, give priority to applying the golden ratio rule to the grid units in the high-frequency gaze area, adjust the center point and boundary position of the grid, generate segmentation ratio data, which includes the coordinate parameters of the golden ratio position and the visual priority ranking of each grid unit, input the segmentation ratio data into the composition optimization module, re-layout the key elements of the simulated adjusted content through the composition optimization algorithm, adjust the center position and relative size of the content elements using the reference position of the golden ratio distribution, give priority to ensuring that the content elements in the high-frequency gaze area comply with the golden ratio layout rule, and visually weaken the auxiliary elements in the edge area. After optimization The result is output as composition data, which includes the optimized content element coordinate information and the adjusted layout ratio. The content element layout in the composition data is processed by the perspective analysis algorithm, the user's line of sight trajectory under different viewing angles is simulated, the clarity, proportion and layout balance of the content elements under each perspective are analyzed, and the line of sight switching path data and the coordinate information of key viewpoints of multiple perspectives are generated to form a perspective data set. The perspective data set includes the viewing direction, distance and content visualization range of each perspective. The perspective data set is dynamically simulated using a three-dimensional scene rendering engine to simulate the visual experience of users when observing screen content from different angles, and the line of sight coverage area and display clarity of each perspective are analyzed. Calculations are performed to generate simulated viewing frame sequence data, record the visual effects and content distribution of each frame, and output as a simulated viewing data set. The simulated viewing data set contains the visual range and clarity evaluation parameters of each viewing angle. An ergonomically based comfort evaluation algorithm is used to analyze the viewing angle parameters recorded in the simulated viewing data set, calculate the user's eye burden and visual adaptability at different viewing angles, and quantitatively evaluate the number of gaze points and the smoothness of the gaze path at each viewing angle. The overall comfort score is calculated in combination with the viewing distance and angle parameters, and the comfort data is output. The comfort data contains the comfort score of each viewing angle and a comprehensive comfort evaluation. The viewing angles with higher scores in the comfort data are screened, and the optimization is used. The viewing angle algorithm adjusts the selected viewing angle parameters, giving priority to enhancing the visual clarity of high-comfort viewing angles, while recalibrating the viewing direction and distance of low-comfort viewing angles. The generated optimization results are stored as optimized viewing angle data, which includes the adjusted viewing angle parameters and viewing angle weight distribution. The optimized viewing angle data and composition data are input into the display effect enhancement module, and the picture is optimized frame by frame through the deep learning-based visual enhancement model to adjust the picture brightness, contrast and color saturation. At the same time, the dynamic texture enhancement algorithm is applied to improve the expressiveness of picture details. The enhanced picture data is stored as effect enhancement data, which includes the optimized picture display parameters and enhanced content frame sequence data. .

[0161] The present invention also provides a computer-TV all-in-one control system for executing the above-mentioned computer-TV all-in-one control method, and the computer-TV all-in-one control system comprises:

[0162] The image acquisition module is used to acquire multi-view images of the integrated machine; perform spatial grid modeling on the multi-view images of the integrated machine to obtain a three-dimensional scene grid; perform scene element analysis on the three-dimensional scene grid to generate a scene element set;

[0163] The environment analysis module is used to track the user's body in the multi-view images of the all-in-one machine to generate a set of user's body images; perform environmental simulation and restoration on the scene element set to obtain the tone of the use environment;

[0164] The content deconstruction module is used to obtain the content displayed on the all-in-one machine; perform content semantic analysis on the content displayed on the all-in-one machine to obtain content semantic data; calculate the interaction intensity of the content semantic data according to the tone of the usage environment to generate content interaction intensity data;

[0165] The operation recognition module is used to infer the body language of the user's body image set based on the tone of the use environment to obtain the user's body language; predict the user's intention based on the user's body language according to the content interaction intensity data, and generate the predicted user operation intention;

[0166] The control adjustment module is used to simulate the content adjustment of the all-in-one display content based on the predicted user operation intention to obtain the simulated adjustment content; optimize the composition of the simulated adjustment content according to the scene element set to generate the all-in-one update content to execute the smart computer TV all-in-one control method.

[0167] The invention provides comprehensive environmental information through the acquisition and processing of multi-view images, ensuring that the system can accurately construct a three-dimensional scene grid. The generation of the three-dimensional scene grid and the analysis of scene elements provide structured data support for subsequent interactions, ensuring that the system can effectively identify and manage various elements in a complex environment. The environmental analysis module can track the user's body movements in real time, and the generated user body image set provides a basis for understanding user behavior. The generation of the use environment tone is restored through environmental simulation, ensuring the adaptability of the system in different environments, and improving the consistency and naturalness of user experience. The content deconstruction module realizes the understanding of the deep meaning of the content through semantic analysis of the displayed content. The interaction intensity calculation is based on the environmental tone, ensuring that the system can quantify the interaction between the user and the content. The operation recognition module can accurately capture the user's needs and emotions through body language speculation and user intention prediction, providing a basis for the system to adjust the content in real time. The control adjustment module ensures that the content display is more in line with the user's visual expectations through simulation adjustment of content and composition optimization, enhancing the effectiveness and attractiveness of information transmission, realizing dynamic understanding and response to user behavior, ensuring that personalized and intelligent experience is provided in a variety of usage scenarios, and enhancing the user stickiness and satisfaction of the smart computer TV all-in-one machine.

[0168] The present invention also provides a computer-TV all-in-one control device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, for executing any one of the computer-TV all-in-one control methods described above.

[0169] The present invention supports complex data processing through efficient computing power, ensuring that the system can execute multiple control tasks in real time, improving the response speed and efficiency of operations, and the memory provides a large-capacity storage space, allowing the system to store multi-view images, user body data and content semantic information, ensuring rapid access and processing of data, and realizing the complete execution of the computer-TV all-in-one control method, ensuring that the functional modules of the system can work together, and improving the overall system performance.

[0170] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0171] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A computer-TV integrated machine control method, characterized in that: The following steps are involved: Step S1: Collect multi-view images of the integrated machine; Perform spatial mesh modeling on the multi-view images of the all-in-one machine to obtain a three-dimensional scene mesh; Perform scene element analysis on the three-dimensional scene grid to generate a scene element set; Step S2: tracking the user's limbs on the multi-view images of the all-in-one device to generate a user limb image set; performing environmental simulation restoration on the scene element set to obtain the use environment tone; Step S3: Obtaining the display content of the integrated machine; Performing content semantic analysis on the contents displayed on the all-in-one machine to obtain content semantic data; calculating the interaction intensity of the content semantic data according to the tone of the usage environment to generate content interaction intensity data; Step S4: inferring the body language of the user's body image set based on the tone of the usage environment to obtain the user's body language; predicting the user's intention based on the user's body language according to the content interaction intensity data to generate a predicted user operation intention; Step S5: adjusting the display content of the all-in-one machine based on the predicted user operation intention to obtain simulated adjustment content; optimizing the composition of the simulated adjustment content according to the scene element set to generate all-in-one machine update content to execute the smart computer TV all-in-one machine control method.

2. The computer-TV all-in-one control method according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: performing multi-angle scanning and acquisition of the surrounding environment of the integrated machine to obtain a multi-view image of the integrated machine; performing multi-view image registration on the multi-view image of the integrated machine to obtain a registered image data set; Step S12: performing point cloud conversion on the registered image data set to obtain a scene point cloud data set; performing grid division on the scene point cloud data set to generate an initial grid data set; Step S13: Optimizing the topological structure of the initial mesh data set to generate a three-dimensional scene mesh; Step S14: performing scene region segmentation on the three-dimensional scene grid to obtain a region labeling data set; performing spatial relationship analysis on the region labeling data set to generate a scene spatial relationship data set; Step S15: extracting scene element attributes from the three-dimensional scene grid to obtain attribute feature data; constructing a hierarchical relationship for the attribute feature data based on the scene spatial relationship data set to generate a scene element set.

3. The computer-TV integrated machine control method according to claim 1, characterized in that: Step S2 includes the following steps: Step S21: Detect skeleton key points on the multi-view image of the integrated device to obtain two-dimensional key point coordinate data; perform spatial projection conversion on the two-dimensional key point coordinate data to generate three-dimensional key point data; Step S22: performing time series correlation analysis on the three-dimensional key point data to obtain key point trajectory data; performing posture reconstruction on the key point trajectory data to generate a user limb image set; Step S23: performing spatial position calculation on the scene element set to obtain element position data; performing spatial topology reconstruction on the scene element set based on the element position data to generate scene topology data; Step S24: performing element restoration mapping on the scene topology data based on the scene element set to obtain the usage environment tone.

4. The computer-TV integrated machine control method according to claim 1, characterized in that: Step S3 includes the following steps: Step S31: obtaining the display content of the integrated machine; performing text recognition and extraction on the display content of the integrated machine to obtain text information data; performing word segmentation processing on the text information data to generate a word-element data set; Step S32: performing part-of-speech tagging analysis on the word-meta data set to obtain part-of-speech tagging data; performing syntactic dependency analysis on the part-of-speech tagging data to obtain grammatical structure data; Step S33: Calculate semantic vectors of the word-metadata set based on the grammatical structure data to generate content semantic data; Step S34: extracting the theme features of the content semantic data to obtain theme feature data; performing scene feature mapping on the usage environment tone according to the theme feature data to generate scene mapping parameters; Step S35: Calculate the interactive parameters of the scene mapping parameters to obtain scene interactive data; perform correlation analysis on the subject feature data and the scene interactive data to generate correlation feature data; Step S36: Perform comprehensive calculation of the interaction strength of the associated feature data to generate content interaction strength data.

5. The computer-TV all-in-one control method according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: positioning joint points of the user's limb image set to obtain joint coordinate data; calculating posture angles of the joint coordinate data to obtain posture angle data; Step S42: performing motion trajectory analysis on the posture angle data to obtain motion trajectory data; performing action pattern recognition on the motion trajectory data to obtain action pattern data; Step S43: performing an environment association analysis on the action mode data based on the usage environment keynote to obtain associated behavior feature data; performing body language inference on the associated behavior feature data to generate user body language; Step S44: Perform interactive analysis on the user's body language based on the content interaction intensity data to obtain interactive feature data; perform user intention prediction on the interactive feature data to generate predicted user operation intention.

6. The computer-TV all-in-one control method according to claim 5, characterized in that: Step S43 includes the following steps: Defining the spatial scope of the use environment keynote to obtain spatial scope data; performing environmental restriction analysis on the spatial scope data based on the use environment keynote to generate environmental restriction data; Extracting scene rules from environmental restriction data to obtain scene rule data; mapping constraint conditions from the scene rule data to generate scene constraint data; Extracting action features from the action pattern data to obtain initial action feature data; performing constraint mapping on the initial action feature data based on the scene constraint data to obtain constraint adjustment data; Optimizing and adjusting the initial behavior characteristic data based on the constraint adjustment data to generate behavior characteristic data; Perform body language analysis on the behavior feature data to obtain action analysis data; perform semantic mapping processing on the action analysis data to generate semantic mapping data; The expression pattern is extracted from the semantic mapping data to obtain the expression pattern data; the expression language of the semantic mapping data is inferred based on the expression pattern data to generate the user's body language.

7. The computer-TV integrated machine control method according to claim 1, characterized in that: Step S5 includes the following steps: Step S51: performing operation content mapping on the predicted user operation intention to obtain simulated operation data; performing content change simulation on the all-in-one machine display content based on the predicted user operation intention based on the simulated operation data to obtain simulated adjustment content; Step S52: Calculate the user visual path for the scene element set to obtain visual path data; Step S53: performing picture composition analysis on the simulated adjustment content based on the visual path data to obtain composition data; performing display effect enhancement processing on the composition data to obtain effect enhancement data; Step S54: optimizing the composition of the simulated adjustment content according to the effect enhancement data, generating all-in-one machine update content, and executing the intelligent computer TV all-in-one machine control method.

8. The computer-TV integrated machine control method according to claim 7, characterized in that: Step S53 includes the following steps: Performing gaze point distribution analysis on the visual path data to obtain gaze distribution data; performing composition grid division on the gaze distribution data to obtain gaze grid layout data; Performing golden section calculation on the gaze grid layout data to generate segmentation ratio data; optimizing the picture composition of the simulated adjustment content based on the segmentation ratio data to obtain composition data; Performing perspective switching analysis on the composition data to obtain a perspective data set; performing viewing perspective simulation on the perspective data set to obtain a simulated viewing data set; Perform comfort calculation on the simulated viewing data set to generate comfort data; perform optimized viewing angle analysis on the viewing angle data set based on the comfort data to generate optimized viewing angle data; The composition data is processed for display effect enhancement according to the optimized viewing angle data to obtain effect enhancement data.

9. A computer-TV all-in-one control system, characterized in that: Used to execute the computer-TV all-in-one control method as claimed in claim 1, the computer-TV all-in-one control system comprises: The image acquisition module is used to acquire multi-view images of the integrated machine; perform spatial grid modeling on the multi-view images of the integrated machine to obtain a three-dimensional scene grid; perform scene element analysis on the three-dimensional scene grid to generate a scene element set; The environment analysis module is used to track the user's body in the multi-view images of the all-in-one machine to generate a set of user's body images; perform environmental simulation and restoration on the scene element set to obtain the tone of the use environment; The content deconstruction module is used to obtain the content displayed on the all-in-one machine; perform content semantic analysis on the content displayed on the all-in-one machine to obtain content semantic data; calculate the interaction intensity of the content semantic data according to the tone of the usage environment to generate content interaction intensity data; The operation recognition module is used to infer the body language of the user's body image set based on the tone of the use environment to obtain the user's body language; predict the user's intention based on the user's body language according to the content interaction intensity data, and generate the predicted user operation intention; The control adjustment module is used to simulate the content adjustment of the all-in-one display content based on the predicted user operation intention to obtain the simulated adjustment content; optimize the composition of the simulated adjustment content according to the scene element set to generate the all-in-one update content to execute the smart computer TV all-in-one control method.

10. A computer-TV integrated machine control device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and executable on the processor, and is used to execute the computer-TV integrated machine control method as claimed in any one of claims 1 to 8.

Images