Intelligent screen interaction method and system based on panoramic nursing and intelligent member service, and medium

Abstract

The invention provides an intelligent screen interaction method and system based on panoramic nursing and intelligent membership service, and a medium, and the method comprises the steps: collecting full-view environment image data of a target region, carrying out the real-time analysis of the environment image data, and obtaining environment monitoring and user behavior characteristics; calling a user portrait database to obtain user identity information, and obtaining user historical habit data and scene preference data; constructing a personalized service model, and generating personalized member content and service information based on the personalized service model; acquiring a user interaction instruction to obtain user execution information; the personalized member content is matched with the service information based on the user execution information to obtain push content, and the push content is transmitted to a smart screen in real time to be displayed; full-view environment coverage is realized by integrating a panoramic camera, accurate recognition of user behaviors and identity information is carried out in cooperation with an image recognition algorithm, family nursing requirements are met, and active pushing of personalized contents and services is realized.

Description

Technical Field

[0001] This application relates to the field of smart screen interaction technology, and more specifically, to a smart screen interaction method, system, and medium based on panoramic care and intelligent membership services. Background Technology

[0002] With the rapid development of smart interactive device technology, smart screens, as core interactive terminals in home scenarios, have been widely used in various fields such as audio-visual entertainment, information display, and home control. Existing smart screen products often focus on expanding single-function scenarios. Some smart screens with basic monitoring functions can only achieve fixed-view image acquisition and simple alarms, lacking full-view environmental coverage and precise behavior analysis capabilities, making it difficult to meet the actual needs of comprehensive home care. At the same time, most smart screen membership services on the market are passively responsive push notifications, providing basic content recommendations based only on users' historical consumption records. They cannot dynamically adjust service strategies based on real-time scenarios and user behavior, resulting in insufficient personalization and adaptability of membership services.

[0003] In addition, the existing smart screens have relatively simple interaction methods, mainly voice or touch control, with low integration of emerging interaction methods such as gesture recognition, and lack of collaborative adaptation capabilities between different interaction modes; the cross-device data synchronization mechanism is imperfect, making it difficult for the smart screen's care data and membership service data to be seamlessly connected with other smart home devices and mobile terminals, forming data silos and failing to achieve deep integration of home care and membership services. Summary of the Invention

[0004] The purpose of this application is to provide a smart screen interaction method, system and medium based on panoramic care and smart membership services. By integrating a panoramic camera to achieve full-view environmental coverage, and combining it with image recognition algorithms to accurately identify user behavior and identity information, it can meet the needs of family care and realize the proactive push of personalized content and services.

[0005] This application also provides a smart screen interaction method based on panoramic care and intelligent membership services, including: Collect full-view environmental image data of the target area, and analyze the environmental image data in real time to obtain environmental monitoring and user behavior characteristics; Access the user profile database to obtain user identity information, and then retrieve user historical habit data and scenario preference data based on the user identity information; A personalized service model is built based on user historical habit data, scenario preference data and user behavior characteristics, and personalized membership content and service information are generated based on the personalized service model. Obtain user interaction commands, and perform voice, touch and gesture recognition based on user interaction commands to obtain user execution information; Based on user execution information, personalized membership content and service information are matched to obtain push content, which is then transmitted to the smart screen for display in real time.

[0006] Optionally, in the smart screen interaction method based on panoramic care and smart membership services described in this application embodiment, the method involves collecting full-view environmental image data of the target area, and performing real-time analysis of the environmental image data to obtain environmental monitoring and user behavior characteristics, specifically including: Collect full-view environmental image data of the target area, and divide the environmental image data into regions based on image segmentation algorithms to obtain several sub-regions; Denoising is performed on each sub-region, and then enhancement is performed on the denoised sub-regions to obtain the enhanced image; The enhanced image of each sub-region is analyzed to separate the environmental region from the user target; Based on user goals, user actions, postures, and activity trajectories are analyzed to obtain user behavior characteristics; Environmental features are extracted based on the environmental region to obtain environmental monitoring data.

[0007] Optionally, in the smart screen interaction method based on panoramic care and smart membership services described in the embodiments of this application, the user profile database is called to obtain user identity information, and user historical habit data and scene preference data are obtained based on the user identity information, specifically including: Collect user facial images, call the user profile database, and match the facial images based on the user profile database to obtain the facial matching degree; Determine whether the facial matching degree is greater than or equal to the set facial matching degree threshold; If it is greater than or equal to, then the user's identity information is obtained, and the user's historical habit data and scenario preference data are obtained based on the user's identity information; The user's historical habit data includes historical service reservation records, usage trajectory of rights and benefits, and cross-device interaction synchronization data; If the value is less than the specified value, then at least one of the following methods will be used to verify the user's identity: account password verification, biometric comparison, or device binding verification.

[0008] Optionally, in the smart screen interaction method based on panoramic care and intelligent membership services described in the embodiments of this application, a personalized service model is constructed based on user historical habit data, scene preference data, and user behavior characteristics. Personalized membership content and service information are generated based on the personalized service model, specifically including: Obtain user profiles, and extract historical habit data, scenario preference data, and user behavior characteristics based on user profiles; Historical habit data, scenario preference data, and user behavior characteristics are anonymized and deduplicated to obtain preprocessed data; Weights are dynamically assigned to preprocessed data based on the analytic hierarchy process. A personalized service model is constructed by fusing data using collaborative filtering and decision tree algorithms, and personalized member content and service information are generated based on this model.

[0009] Optionally, in the smart screen interaction method based on panoramic care and smart membership services described in the embodiments of this application, user interaction commands are obtained, and voice, touch, and gesture recognition are performed based on the user interaction commands to obtain user execution information, specifically including: Collect user voice commands, touch commands, and gesture commands, and simultaneously label the trigger timestamps and collection device identifiers of the voice commands, touch commands, and gesture commands; User voice commands, touch commands, and gesture commands are categorized to obtain different types of voice commands, touch commands, and gesture commands; The voice commands are denoised and semantically parsed. The parsing results are adjusted based on the user's dialect habits and command expression habits. The parsing results are then converted into standardized text commands to obtain the speech recognition results. The touch recognition algorithm analyzes touch commands to identify the user's touch position, touch force, touch duration and continuous operation trajectory, and matches the corresponding touch command logic to obtain the touch recognition result. Gesture contours, motion trajectories, and posture features are extracted based on gesture commands and compared with a preset gesture library to obtain gesture recognition results; The voice recognition results, touch recognition results and gesture recognition results are cross-validated. If there is a conflict between multimodal commands, the valid commands are selected based on the user's preset interaction priority rules. Parse the user intent corresponding to the valid command and generate structured user execution information.

[0010] Optionally, in the smart screen interaction method based on panoramic care and smart membership services described in the embodiments of this application, personalized membership content and service information are matched based on user execution information to obtain push content, and the push content is transmitted to the smart screen for display in real time, specifically including: Obtain user execution information, and analyze the instruction type, execution object, and execution parameters based on the user execution information; The matching results are obtained by matching the instruction type, execution object, and execution parameters with personalized member content and service information. Correction information is generated based on the user's historical habits and current scenario preferences. The matching results are then adjusted based on the correction information to obtain the push content. The pushed content is transmitted to the smart screen in real time for display, and the display position and display method are automatically adjusted based on the user's interaction habits.

[0011] Secondly, embodiments of this application provide a smart screen interaction system based on panoramic care and intelligent membership services. The system includes a memory and a processor. The memory includes a program for a smart screen interaction method based on panoramic care and intelligent membership services. When the program for the smart screen interaction method based on panoramic care and intelligent membership services is executed by the processor, it implements the following steps: Collect full-view environmental image data of the target area, and analyze the environmental image data in real time to obtain environmental monitoring and user behavior characteristics; Access the user profile database to obtain user identity information, and then retrieve user historical habit data and scenario preference data based on the user identity information; A personalized service model is built based on user historical habit data, scenario preference data and user behavior characteristics, and personalized membership content and service information are generated based on the personalized service model. Obtain user interaction commands, and perform voice, touch and gesture recognition based on user interaction commands to obtain user execution information; Based on user execution information, personalized membership content and service information are matched to obtain push content, which is then transmitted to the smart screen for display in real time.

[0012] Optionally, in the smart screen interaction system based on panoramic monitoring and intelligent membership services described in this application embodiment, full-view environmental image data of the target area is collected, and the environmental image data is analyzed in real time to obtain environmental monitoring and user behavior characteristics, specifically including: Collect full-view environmental image data of the target area, and divide the environmental image data into regions based on image segmentation algorithms to obtain several sub-regions; Denoising is performed on each sub-region, and then enhancement is performed on the denoised sub-regions to obtain the enhanced image; The enhanced image of each sub-region is analyzed to separate the environmental region from the user target; Based on user goals, user actions, postures, and activity trajectories are analyzed to obtain user behavior characteristics; Environmental features are extracted based on the environmental region to obtain environmental monitoring data.

[0013] Optionally, in the smart screen interaction system based on panoramic care and intelligent membership services described in this application embodiment, the user profile database is called to obtain user identity information, and user historical habit data and scene preference data are obtained based on the user identity information, specifically including: Collect user facial images, call the user profile database, and match the facial images based on the user profile database to obtain the facial matching degree; Determine whether the facial matching degree is greater than or equal to the set facial matching degree threshold; If it is greater than or equal to, then the user's identity information is obtained, and the user's historical habit data and scenario preference data are obtained based on the user's identity information; The user's historical habit data includes historical service reservation records, usage trajectory of rights and benefits, and cross-device interaction synchronization data; If the value is less than the specified value, then at least one of the following methods will be used to verify the user's identity: account password verification, biometric comparison, or device binding verification.

[0014] Thirdly, embodiments of this application also provide a computer-readable storage medium, which includes a smart screen interaction method program based on panoramic care and smart membership services. When the smart screen interaction method program based on panoramic care and smart membership services is executed by a processor, it implements the steps of the smart screen interaction method based on panoramic care and smart membership services as described in any of the above claims.

[0015] As can be seen from the above, the smart screen interaction method, system, and medium based on panoramic care and intelligent membership services provided in this application collect full-view environmental image data of the target area, analyze the environmental image data in real time to obtain environmental monitoring and user behavior characteristics; call the user profile database to obtain user identity information, and obtain user historical habit data and scene preference data based on user identity information; construct a personalized service model based on user historical habit data, scene preference data, and user behavior characteristics, and generate personalized membership content and service information based on the personalized service model; obtain user interaction commands, and perform voice, touch, and gesture recognition based on user interaction commands to obtain user execution information; match personalized membership content and service information based on user execution information to obtain push content, and transmit the push content to the smart screen for display in real time; achieve full-view environmental coverage by integrating a panoramic camera, and accurately identify user behavior and identity information by combining with image recognition algorithms to meet family care needs and realize the proactive push of personalized content and services. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a smart screen interaction method based on panoramic care and smart membership services provided in this application embodiment; Figure 2A flowchart illustrating the environmental image data parsing process of the smart screen interaction method based on panoramic care and smart membership services provided in this application embodiment; Figure 3 This is a schematic diagram of the smart screen structure for the smart screen interaction method based on panoramic care and smart membership services provided in the embodiments of this application.

[0018] In the picture, 1 is the camera, 2 is the touch panel, 3 is the microphone, and 4 is the base. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0020] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0021] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a smart screen interaction method based on panoramic care and smart membership services, as described in some embodiments of this application. This smart screen interaction method based on panoramic care and smart membership services is used in a terminal device and includes the following steps: S101: Collects full-view environmental image data of the target area, performs real-time analysis of the environmental image data, and obtains environmental monitoring and user behavior characteristics; S102, call the user profile database to obtain user identity information, and obtain user historical habit data and scenario preference data based on user identity information; S103, build a personalized service model based on user historical habit data, scenario preference data and user behavior characteristics, and generate personalized membership content and service information based on the personalized service model; S104, Obtain user interaction commands, perform voice, touch and gesture recognition based on user interaction commands, and obtain user execution information; S105 matches personalized membership content and service information based on user execution information to obtain push content, and transmits the push content to the smart screen for display in real time.

[0022] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating the environmental image data parsing process of a smart screen interaction method based on panoramic monitoring and intelligent membership services, as described in some embodiments of this application. According to embodiments of the present invention, full-view environmental image data of the target area is collected, and the environmental image data is analyzed in real time to obtain environmental monitoring and user behavior characteristics, specifically including: S201: Collect full-view environmental image data of the target area, and divide the environmental image data into regions based on the image segmentation algorithm to obtain several sub-regions; S202, Denoise each sub-region and then enhance the denoised sub-regions to obtain an enhanced image; S203, analyze the enhanced image of each sub-region and divide the environment region and user target; S204, based on user goals, analyze user actions, postures and activity trajectories to obtain user behavior characteristics; S205, based on environmental region, extracts environmental features to obtain environmental monitoring data.

[0023] It should be noted that a fisheye lens or multi-camera stitching scheme is used to achieve 360° coverage of the target area without blind spots. The collected full-view environmental image data undergoes a real-time analysis process. First, the image optimization unit sequentially completes noise reduction, enhancement, and image segmentation processing to separate the environmental background from the user target. Then, the built-in behavior recognition model extracts features and performs intelligent analysis on the user target, simultaneously obtaining environmental monitoring results and user behavior characteristics. Finally, a clear environmental monitoring screen and structured analysis data are output. Specifically, the environmental monitoring results include real-time images of the target area, dynamic change information within the area, and abnormal event warning signals. The user behavior characteristics include user actions and postures (such as standing, sitting, walking, falling), movement trajectories (such as activity range and movement paths within the area), activity area distribution, frequency and duration of behavior occurrences, and interactive behaviors (such as touching the smart screen and voice wake-up), providing accurate data support for subsequent personalized service adaptation and scene determination.

[0024] According to an embodiment of the present invention, a user profile database is invoked to obtain user identity information, and user historical habit data and scenario preference data are obtained based on the user identity information, specifically including: Collect user facial images, call the user profile database, and match the facial images based on the user profile database to obtain the facial matching degree; Determine whether the facial matching degree is greater than or equal to the set facial matching degree threshold; If it is greater than or equal to, then the user's identity information is obtained, and the user's historical habit data and scenario preference data are obtained based on the user's identity information; User history data includes historical service reservation records, usage patterns of benefits, and cross-device interaction synchronization data; If the value is less than the specified value, then at least one of the following methods will be used to verify the user's identity: account password verification, biometric comparison, or device binding verification.

[0025] It should be noted that after sending the identity verification request, the user's identity is confirmed through at least one of the following methods: account password verification, biometric comparison, or device binding verification, to ensure the security and accuracy of data acquisition. The acquired user historical habit data also includes historical service appointment records, rights usage trajectory, and cross-device interaction synchronization data. The scenario preference data is further refined into scenario time preferences (such as preference for leisure scenario services on weekday evenings and preference for family care scenario services on weekends), scenario environment preferences (such as preference for audio-visual services in bright environments and preference for voice broadcast services in dark environments), and scenario-related group preferences (such as preference for personalized services in solitary scenarios and preference for family sharing services in parent-child scenarios). All data is imported into the personalized service model after being anonymized to ensure user privacy and security while improving the model adaptation accuracy.

[0026] According to embodiments of the present invention, a personalized service model is constructed based on user historical habit data, scenario preference data, and user behavior characteristics. Personalized membership content and service information are generated based on this personalized service model, specifically including: Obtain user profiles, and extract historical habit data, scenario preference data, and user behavior characteristics based on user profiles; Historical habit data, scenario preference data, and user behavior characteristics are anonymized and deduplicated to obtain preprocessed data; Weights are dynamically assigned to preprocessed data based on the analytic hierarchy process. A personalized service model is constructed by fusing data using collaborative filtering and decision tree algorithms, and personalized member content and service information are generated based on this model.

[0027] It should be noted that the construction of a personalized service model includes: 1. Data preprocessing: The extracted user historical habit data and scenario preference data are anonymized, deduplicated, standardized, and outlier removed to avoid invalid data from interfering with the model accuracy. At the same time, real-time user behavior features are classified and labeled by timestamp and behavior correlation, and transformed into structured data consistent with the basic data format to achieve dimensional alignment between the basic data and real-time features and ensure data compatibility.

[0028] 2. Dynamic Weight Allocation: A multi-dimensional weight system is constructed using the analytic hierarchy process (AHP). Weights are allocated based on data timeliness and the impact on demand. Historical user habit data accounts for 40%-50%, with long-term stable core preferences having higher weights than temporary behaviors, reinforcing the anchoring effect of basic preferences. Scenario preference data accounts for 30%-40%, with adaptive weight increases for preferences that highly match the current scenario, accurately adapting to real-time scenario needs. Real-time user behavior characteristics account for 10%-20%, with their dynamic adjustment coefficient fluctuating based on the duration of the behavior and its relevance to the scenario, efficiently capturing users' immediate needs.

[0029] 3. Model Training and Validation: Based on collaborative filtering algorithms and decision tree models in machine learning, the preprocessed basic data is deeply integrated with real-time behavioral features to construct a dual-core personalized service model of "basic preference anchoring + real-time scenario dynamic adaptation". Using users' historical interaction data as training samples and real-time behavioral features as the basis for dynamic adjustment, the model outputs the core service demand tags, content preference dimensions, service push priority ranking, and device linkage logic in the current user scenario. The built-in adaptation verification unit performs real-time verification of the matching degree between the model output results and the user's historical behavior and scenario attributes. When the matching degree is lower than a preset threshold (such as 60%), secondary feature fusion and parameter calibration are automatically triggered to ensure the accuracy of the model output.

[0030] Personalized membership content generation includes: 1. Customized Information Content: Based on users' areas of interest, historical browsing history, and current scenario, generate exclusive special topics, industry news, and scenario-based guides (such as parenting information for parent-child scenarios and recipe guides for kitchen scenarios), and simultaneously mark the timeliness, source, and related content of the information to make it easy for users to quickly obtain core information.

[0031] 2. Exclusive Benefits: By linking user membership level, points balance, and preferred scenarios, targeted push notifications will be sent for points redemption benefits, limited-time exclusive offers, level upgrade benefits, and expiration reminders. The usage conditions, validity period, and redemption path of the benefits will be clearly defined to improve the accuracy of benefit delivery.

[0032] 3. Contextualized audio-visual content: Adapt to the current scene environment parameters (such as brightness and quietness) and user audio-visual preferences to push content such as movies, music, audiobooks, and podcasts, and automatically adapt the playback clarity and volume mode according to the scene to meet the usage needs of the scene.

[0033] 4. Customized content: For different scenarios such as parent-child activities, family gatherings, and life services, we push exclusive interactive materials (parent-child games, family photo album templates) and process guides (housekeeping service procedures, repair precautions) to adapt to the needs of different scenarios.

[0034] Personalized service information generation includes: 1. Benefits Redemption Service Information: Provides a dedicated channel for redeeming physical goods, virtual content (membership time, paid content), and service vouchers with points, supports one-click redemption, and provides logistics tracking (physical benefits) or arrival notification (virtual benefits) functions.

[0035] 2. Customized Appointment Service Information: Based on the current scenario needs, push suitable services such as housekeeping, early childhood education, repair, and physical examination, link users' frequently used service provider preferences, display service prices, available time slots and user rating system, support one-click appointment, rescheduling and cancellation operations, and improve service convenience.

[0036] 3. Cross-device linkage service information: Based on the interaction preferences of scene devices, generate linkage service instructions between smart screen and related devices (such as linking children's toys to play sound effects in parent-child scenes, or linking door and window sensors to trigger early warnings in security scenes), to achieve multi-device collaborative services.

[0037] 4. Contextualized security service information: For home care scenarios, push personalized security early warning services (customized abnormal behavior alarms, area intrusion reminders) and linkage response plans (automatic recording, associated terminal alarms) to enhance home care capabilities.

[0038] 5. Tier-Specific Service Information: Provide differentiated service information based on membership level (higher-level members enjoy dedicated customer service, priority access to new services, and service fee discounts), clearly defining the boundaries of rights and benefits for each level and ensuring the differentiated implementation of membership rights.

[0039] According to an embodiment of the present invention, user interaction commands are obtained, and voice, touch, and gesture recognition are performed based on the user interaction commands to obtain user execution information, specifically including: Collect user voice commands, touch commands, and gesture commands, and simultaneously label the trigger timestamps and collection device identifiers of the voice commands, touch commands, and gesture commands; User voice commands, touch commands, and gesture commands are categorized to obtain different types of voice commands, touch commands, and gesture commands; The voice commands are denoised and semantically parsed. The parsing results are adjusted based on the user's dialect habits and command expression habits. The parsing results are then converted into standardized text commands to obtain the speech recognition results. The touch recognition algorithm analyzes touch commands to identify the user's touch position, touch force, touch duration and continuous operation trajectory, and matches the corresponding touch command logic to obtain the touch recognition result. Gesture contours, motion trajectories, and posture features are extracted based on gesture commands and compared with a preset gesture library to obtain gesture recognition results; The voice recognition results, touch recognition results and gesture recognition results are cross-validated. If there is a conflict between multimodal commands, the valid commands are selected based on the user's preset interaction priority rules. Parse the user intent corresponding to the valid command and generate structured user execution information.

[0040] It should be noted that, as Figure 3 As shown, the smart screen includes a microphone 3, a touch panel 2, and a camera 1, with a base 4 at the bottom for supporting the placement of the smart screen. The microphone, touch panel, and camera respectively collect user voice commands, touch commands, and gesture commands, and simultaneously mark the command trigger timestamp and the collection device identifier. The commands are initially classified according to their type (control, query, switching, and setting), and invalid commands such as environmental noise and accidental touches are filtered out. The multimodal recognition process is as follows: In the speech recognition stage, the speech recognition unit performs noise reduction and semantic analysis on the collected speech commands, and optimizes the recognition accuracy by combining the user's dialect habits and command expression habits, converting them into standardized text commands; in the touch recognition stage, the touch sensing unit identifies the user's touch position, touch force, touch duration, and continuous operation trajectory, and matches the corresponding touch command logic; in the gesture recognition stage, combined with the collected environmental image data, the gesture recognition unit extracts the gesture contour, motion trajectory, and posture features, and compares them with a preset gesture library (such as swiping, clicking, and clenching a fist) to complete the recognition. The execution information integration and verification includes: cross-validating the multimodal recognition results; if there are multimodal command conflicts, determining the valid command based on the user's preset interaction priority rules and eliminating contradictory commands; then parsing the user intent corresponding to the valid command to generate structured user execution information, which specifically includes command type, execution object (target content / service), operation action, execution parameters (such as playback progress, volume, screen switching amplitude) and command priority.

[0041] The output and feedback of execution information includes: synchronizing the integrated user execution information to the main control module and the corresponding service module to trigger subsequent operation execution; and providing feedback to the user on the recognition results and execution status through voice broadcast, screen prompts and light indicators via the interactive feedback unit to ensure that the user is aware of the instruction processing status.

[0042] According to an embodiment of the present invention, personalized membership content and service information are matched based on user execution information to obtain push content, and the push content is transmitted to a smart screen for display in real time, specifically including: Obtain user execution information, and analyze the instruction type, execution object, and execution parameters based on the user execution information; The matching results are obtained by matching the instruction type, execution object, and execution parameters with personalized member content and service information. Correction information is generated based on the user's historical habits and current scenario preferences. The matching results are then adjusted based on the correction information to obtain the push content. The pushed content is transmitted to the smart screen in real time for display, and the display position and display method are automatically adjusted based on the user's interaction habits.

[0043] It should be noted that the instruction type, execution object and parameters are extracted from the user's execution information and compared and matched with the tags and attributes of personalized membership content and service information. Priority is given to associating the content / service corresponding to the user's explicit instructions. At the same time, the user's historical habits and current scenario preferences are combined to supplement and adapt recommendations, and invalid and mismatched items are filtered out. Push content integration: Sorted according to the principle of "prioritizing content corresponding to core instructions and supplementing with recommendations", the content is integrated into a structured push content, with core information, access path and operation guide marked, and the layout optimized to adapt to the display specifications of smart screens; Real-time transmission and display: The integrated push content is transmitted to the smart screen in real time through the data transmission channel scheduled by the main control module, triggering screen display commands and supporting automatic adjustment of display position and display mode according to user interaction habits; Display feedback: Synchronously collect smart screen display status data, and after confirming that the content is displayed normally, prompt the user through the interactive feedback unit. If a display abnormality occurs, automatically retry the transmission and issue an alarm.

[0044] In summary, compared with the prior art, the present invention has the following beneficial effects: 1. Enhance the comprehensiveness and accuracy of home care: By integrating panoramic cameras to achieve full-view environmental coverage, and combining with image recognition algorithms to accurately identify abnormal events, user behavior and identity information, it breaks through the limitations of existing smart screens with fixed-view monitoring and simple alarms, and can meet the needs of home care in all aspects. At the same time, through a multi-dimensional early warning mechanism, it further protects the safety of the home environment.

[0045] 2. Enhance the personalization and proactivity of membership services: The intelligent membership service module combines real-time scenarios, user behavior data, and historical preferences to proactively push personalized content and services, abandoning the traditional passive response recommendation model. At the same time, it optimizes the recommendation model through machine learning, dynamically adjusts service strategies, improves the adaptability of membership services and user stickiness, and highlights the value of membership services.

[0046] 3. Optimize the convenience and flexibility of user interaction experience: Integrate multimodal interaction methods such as voice, touch and gesture recognition, support static and dynamic gesture recognition, dialect recognition and multimodal command fusion response, solve the problems of single interaction methods and poor coordination of existing smart screens, adapt to different user habits and usage scenarios, reduce the operation threshold and improve interaction efficiency.

[0047] 4. Achieve deep integration and sharing of services and data: Through an encrypted cross-device data synchronization mechanism, seamless connection of care data, interaction records, and membership data between smart screens and associated devices is achieved, breaking down data silos and realizing deep integration of home care and membership services. At the same time, privacy protection measures such as data anonymization and encrypted storage are used to balance data sharing and user privacy and security.

[0048] 5. Enhance the scene adaptability and integration value of smart screens: This invention integrates panoramic monitoring, intelligent membership services, multimodal interaction and cross-device synchronization functions, which greatly improves the functional integration of smart screens, making them flexibly adaptable to various family scenarios such as audio-visual entertainment, home security, and daily management. It breaks through the single-function positioning of existing smart screens, improves the efficiency and comprehensive value of device use, and promotes the deep application of smart interactive devices in family scenarios.

[0049] Secondly, embodiments of this application provide a smart screen interaction system based on panoramic care and smart membership services. The system includes a memory and a processor. The memory includes a program for a smart screen interaction method based on panoramic care and smart membership services. When the program is executed by the processor, it implements the following steps: Collect full-view environmental image data of the target area, analyze the environmental image data in real time, and obtain environmental monitoring and user behavior characteristics; Access the user profile database to obtain user identity information, and then retrieve user historical habit data and scenario preference data based on the user identity information; A personalized service model is built based on user historical habit data, scenario preference data and user behavior characteristics, and personalized membership content and service information are generated based on the personalized service model. Obtain user interaction commands, and perform voice, touch and gesture recognition based on user interaction commands to obtain user execution information; Based on user execution information, personalized membership content and service information are matched to obtain push content, which is then transmitted to the smart screen for display in real time.

[0050] It should be noted that the system supports multimodal interaction, including voice, touch, and gesture recognition, and has cross-device data synchronization capabilities, enabling seamless integration of home care and membership services. This invention enhances the scene adaptability and service integration of smart screens, improving user experience and device utilization efficiency.

[0051] According to an embodiment of the present invention, full-view environmental image data of the target area is collected, and the environmental image data is analyzed in real time to obtain environmental monitoring and user behavior characteristics, specifically including: Collect full-view environmental image data of the target area, and divide the environmental image data into regions based on image segmentation algorithms to obtain several sub-regions; Denoising is performed on each sub-region, and then enhancement is performed on the denoised sub-regions to obtain the enhanced image; The enhanced image of each sub-region is analyzed to separate the environmental region from the user target; Based on user goals, user actions, postures, and activity trajectories are analyzed to obtain user behavior characteristics; Environmental features are extracted based on the environmental region to obtain environmental monitoring data.

[0052] According to an embodiment of the present invention, a user profile database is invoked to obtain user identity information, and user historical habit data and scenario preference data are obtained based on the user identity information, specifically including: Collect user facial images, call the user profile database, and match the facial images based on the user profile database to obtain the facial matching degree; Determine whether the facial matching degree is greater than or equal to the set facial matching degree threshold; If it is greater than or equal to, then the user's identity information is obtained, and the user's historical habit data and scenario preference data are obtained based on the user's identity information; User history data includes historical service reservation records, usage patterns of benefits, and cross-device interaction synchronization data; If the value is less than the specified value, then at least one of the following methods will be used to verify the user's identity: account password verification, biometric comparison, or device binding verification.

[0053] A third aspect of the present invention provides a computer-readable storage medium including a smart screen interaction method program based on panoramic care and smart membership services. When the smart screen interaction method program based on panoramic care and smart membership services is executed by a processor, it implements the steps of the smart screen interaction method based on panoramic care and smart membership services as described above.

[0054] This invention discloses a smart screen interaction method, system, and medium based on panoramic monitoring and intelligent membership services. It collects full-view environmental image data of the target area, performs real-time analysis of the environmental image data to obtain environmental monitoring and user behavior characteristics; calls a user profile database to obtain user identity information, and obtains user historical habit data and scene preference data based on the user identity information; constructs a personalized service model based on the user historical habit data, scene preference data, and user behavior characteristics, and generates personalized membership content and service information based on the personalized service model; obtains user interaction commands, performs voice, touch, and gesture recognition based on the user interaction commands to obtain user execution information; matches personalized membership content and service information based on user execution information to obtain push content, and transmits the push content to the smart screen for display in real time; integrates a panoramic camera to achieve full-view environmental coverage, and combines it with image recognition algorithms for accurate identification of user behavior and identity information, meeting family care needs and realizing proactive push of personalized content and services.

[0055] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0056] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0057] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0058] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0059] Alternatively, if the integrated units of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A smart screen interaction method based on panoramic monitoring and intelligent membership services, characterized in that, include: Collect full-view environmental image data of the target area, and analyze the environmental image data in real time to obtain environmental monitoring and user behavior characteristics; Access the user profile database to obtain user identity information, and then retrieve user historical habit data and scenario preference data based on the user identity information; A personalized service model is built based on user historical habit data, scenario preference data and user behavior characteristics, and personalized membership content and service information are generated based on the personalized service model. Obtain user interaction commands, and perform voice, touch and gesture recognition based on user interaction commands to obtain user execution information; Based on user execution information, personalized membership content and service information are matched to obtain push content, which is then transmitted to the smart screen for display in real time.

2. The intelligent screen interaction method based on panoramic monitoring and intelligent membership services according to claim 1, characterized in that, Collect full-view environmental image data of the target area, and perform real-time analysis of the environmental image data to obtain environmental monitoring and user behavior characteristics, specifically including: Collect full-view environmental image data of the target area, and divide the environmental image data into regions based on image segmentation algorithms to obtain several sub-regions; Denoising is performed on each sub-region, and then enhancement is performed on the denoised sub-regions to obtain the enhanced image; The enhanced image of each sub-region is analyzed to separate the environmental region from the user target; Based on user goals, user actions, postures, and activity trajectories are analyzed to obtain user behavior characteristics; Environmental features are extracted based on the environmental region to obtain environmental monitoring data.

3. The intelligent screen interaction method based on panoramic monitoring and intelligent membership services according to claim 2, characterized in that, Access the user profile database to obtain user identity information, and then retrieve user historical habit data and scenario preference data based on this identity information. Specifically, this includes: Collect user facial images, call the user profile database, and match the facial images based on the user profile database to obtain the facial matching degree; Determine whether the facial matching degree is greater than or equal to the set facial matching degree threshold; If it is greater than or equal to, then the user's identity information is obtained, and the user's historical habit data and scenario preference data are obtained based on the user's identity information; The user's historical habit data includes historical service reservation records, usage trajectory of rights and benefits, and cross-device interaction synchronization data; If the value is less than the specified value, then at least one of the following methods will be used to verify the user's identity: account password verification, biometric comparison, or device binding verification.

4. The intelligent screen interaction method based on panoramic monitoring and intelligent membership services according to claim 3, characterized in that, A personalized service model is built based on user historical habit data, scenario preference data, and user behavior characteristics. Personalized membership content and service information are generated based on this model, specifically including: Obtain user profiles, and extract historical habit data, scenario preference data, and user behavior characteristics based on user profiles; Historical habit data, scenario preference data, and user behavior characteristics are anonymized and deduplicated to obtain preprocessed data; Weights are dynamically assigned to preprocessed data based on the analytic hierarchy process. A personalized service model is constructed by fusing data using collaborative filtering and decision tree algorithms, and personalized member content and service information are generated based on this model.

5. The intelligent screen interaction method based on panoramic care and intelligent membership services according to claim 4, characterized in that, Obtain user interaction commands, and perform voice, touch, and gesture recognition based on these commands to obtain user execution information, specifically including: Collect user voice commands, touch commands, and gesture commands, and simultaneously label the trigger timestamps and collection device identifiers of the voice commands, touch commands, and gesture commands; User voice commands, touch commands, and gesture commands are categorized to obtain different types of voice commands, touch commands, and gesture commands; The voice commands are denoised and semantically parsed. The parsing results are adjusted based on the user's dialect habits and command expression habits. The parsing results are then converted into standardized text commands to obtain the speech recognition results. The touch recognition algorithm analyzes touch commands to identify the user's touch position, touch force, touch duration and continuous operation trajectory, and matches the corresponding touch command logic to obtain the touch recognition result. Gesture contours, motion trajectories, and posture features are extracted based on gesture commands and compared with a preset gesture library to obtain gesture recognition results; The voice recognition results, touch recognition results and gesture recognition results are cross-validated. If there is a conflict between multimodal commands, the valid commands are selected based on the user's preset interaction priority rules. Parse the user intent corresponding to the valid command and generate structured user execution information.

6. The intelligent screen interaction method based on panoramic care and intelligent membership services according to claim 5, characterized in that, Based on user execution information, personalized membership content and service information are matched to obtain push content, which is then transmitted to the smart screen for display in real time. Specifically, this includes: Obtain user execution information, and analyze the instruction type, execution object, and execution parameters based on the user execution information; The matching results are obtained by matching the instruction type, execution object, and execution parameters with personalized member content and service information. Correction information is generated based on the user's historical habits and current scenario preferences. The matching results are then adjusted based on the correction information to obtain the push content. The pushed content is transmitted to the smart screen in real time for display, and the display position and display method are automatically adjusted based on the user's interaction habits.

7. A smart screen interactive system based on panoramic monitoring and intelligent membership services, characterized in that, The system includes a memory and a processor. The memory includes a program for a smart screen interaction method based on panoramic care and smart membership services. When the program for the smart screen interaction method based on panoramic care and smart membership services is executed by the processor, it performs the following steps: Collect full-view environmental image data of the target area, and analyze the environmental image data in real time to obtain environmental monitoring and user behavior characteristics; Access the user profile database to obtain user identity information, and then retrieve user historical habit data and scenario preference data based on the user identity information; A personalized service model is built based on user historical habit data, scenario preference data and user behavior characteristics, and personalized membership content and service information are generated based on the personalized service model. Obtain user interaction commands, and perform voice, touch and gesture recognition based on user interaction commands to obtain user execution information; Based on user execution information, personalized membership content and service information are matched to obtain push content, which is then transmitted to the smart screen for display in real time.

8. The intelligent screen interaction system based on panoramic care and intelligent membership services according to claim 7, characterized in that, Collect full-view environmental image data of the target area, and perform real-time analysis of the environmental image data to obtain environmental monitoring and user behavior characteristics, specifically including: Collect full-view environmental image data of the target area, and divide the environmental image data into regions based on image segmentation algorithms to obtain several sub-regions; Denoising is performed on each sub-region, and then enhancement is performed on the denoised sub-regions to obtain the enhanced image; The enhanced image of each sub-region is analyzed to separate the environmental region from the user target; Based on user goals, user actions, postures, and activity trajectories are analyzed to obtain user behavior characteristics; Environmental features are extracted based on the environmental region to obtain environmental monitoring data.

9. The intelligent screen interaction system based on panoramic care and intelligent membership services according to claim 8, characterized in that, Access the user profile database to obtain user identity information, and then retrieve user historical habit data and scenario preference data based on this identity information. Specifically, this includes: Collect user facial images, call the user profile database, and match the facial images based on the user profile database to obtain the facial matching degree; Determine whether the facial matching degree is greater than or equal to the set facial matching degree threshold; If it is greater than or equal to, then the user's identity information is obtained, and the user's historical habit data and scenario preference data are obtained based on the user's identity information; The user's historical habit data includes historical service reservation records, usage trajectory of rights and benefits, and cross-device interaction synchronization data; If the value is less than the specified value, then at least one of the following methods will be used to verify the user's identity: account password verification, biometric comparison, or device binding verification.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a smart screen interaction method program based on panoramic care and smart membership services. When the smart screen interaction method program based on panoramic care and smart membership services is executed by a processor, it implements the steps of the smart screen interaction method based on panoramic care and smart membership services as described in any one of claims 1 to 6.

Images