Visual communication information processing system and method based on Internet of Things

Abstract

The invention provides a visual communication information processing system and method based on Internet of Things, and aims to solve the problems of static visual communication content, poor accuracy and lack of dynamic adaptation and feedback optimization capability in the prior art. The system comprises a data acquisition module, a data processing module, a visual information generation module, an information transmission module, a display module and a feedback module. Environmental data and user behavior data are collected in real time through an Internet of Things sensor, a data processing module carries out analysis according to an environmental characteristic value and a user behavior characteristic value and generates a priority decision, a visual information generation module dynamically generates content in combination with an algorithm, and an information transmission module sends the content to a display module for display. Meanwhile, a feedback module collects user interaction data, and content correlation scores and algorithm parameters are optimized in real time. According to the method, dynamic adaptation, accurate pushing and self-adaptive optimization of visual contents can be realized, and the method is widely applied to the fields of advertisement putting, public service, culture exhibition, intelligent navigation and the like.

Description

Technical Field

[0001] The present invention relates to the field of visual communication technology, and in particular to a visual communication information processing system and method based on the Internet of Things. Background Art

[0002] As an important means of information dissemination, visual communication technology has been widely used in advertising, public services, industrial monitoring, cultural display and other fields. However, the existing technology still has many deficiencies in practical applications and cannot meet the needs of dynamic and intelligent information interaction. The following analyzes the limitations of the existing technology from multiple aspects:

[0003] Most current visual communication systems are based on preset static content. The display content is usually designed and fixed in advance, and cannot be dynamically adjusted according to real-time changes in the environment. For example, in outdoor billboards, the display content cannot be dynamically adjusted according to weather changes, light intensity, or crowd density. This static design leads to low efficiency in information dissemination and cannot maximize user attention or provide accurate services.

[0004] Traditional visual communication systems cannot effectively capture and analyze user behavior data, such as the user's gaze direction, dwell time, gesture operation, etc. Even if simple user interaction data can be collected in some scenarios, this data has not been fully utilized to optimize content display. For example, on the advertising screen in a shopping mall, different users may be interested in different products, but the existing system cannot identify the user's personalized needs, nor can it dynamically adjust the display content according to user behavior.

[0005] Although the existing visual communication system has gradually realized the networking function in hardware, its integration with the Internet of Things technology is relatively low. IoT sensors can collect environmental data (such as temperature and humidity, light intensity, crowd density, etc.) in real time, but these data lack a unified processing and analysis mechanism in the existing system, and fail to provide data support for the dynamic generation of visual communication content. For example, in museums or exhibition halls, although there are cameras and sensors, these devices often work independently and cannot collaboratively perceive and process environmental data and user behavior.

[0006] Current visual communication systems usually use simple rules or manually designed logic to select content, and are unable to select the most suitable content for the current scenario from a large amount of candidate information. For example, at public transportation stations, the system cannot intelligently determine whether passengers need real-time arrival information or weather reminders, and often displays content in a fixed carousel format, which wastes resources and reduces user satisfaction.

[0007] Existing systems perform poorly in collecting and utilizing user feedback. They usually only record simple user behavior data, but are unable to conduct in-depth analysis of this data to optimize content selection strategies. For example, in the navigation system of a large shopping mall, user behaviors such as clicks and gaze time are not effectively used to optimize navigation paths or adjust recommended content. Such a system that lacks an adaptive feedback mechanism is difficult to achieve continuous optimization and improvement.

[0008] Existing visual communication systems can usually only process a single data source and cannot integrate multi-dimensional data for in-depth analysis and mining. For example, when processing environmental data (such as light, temperature and humidity) and user behavior data (such as gaze direction and dwell time) at the same time, the system lacks a unified calculation model and algorithm, making it difficult to integrate multi-dimensional information to generate accurate visual content.

[0009] In summary, the existing visual communication system has the problems of being static and single, lacking the ability to dynamically perceive and analyze the environment and user behavior, and failing to fully utilize the advantages of the Internet of Things technology. In particular, the existing technology shows obvious deficiencies in terms of dynamic content generation, accurate content selection, and user feedback optimization. These problems limit the effect of visual communication technology in actual application scenarios and cannot meet the complex and diverse needs of modern information dissemination. Therefore, we urgently need to design a visual communication information processing system based on the Internet of Things and its method to solve the above problems. Summary of the invention

[0010] The present invention provides a visual communication information processing system and method based on the Internet of Things, which are used to solve the defects in the prior art.

[0011] A visual communication information processing system based on the Internet of Things, including the following modules:

[0012] Data collection module: obtains environmental data and user behavior data through the IoT sensor network. The environmental data includes physical parameters such as temperature, humidity, light intensity, and sound, and the user behavior data includes gaze direction, dwell time, gestures, etc.;

[0013] Data processing module: processes the raw data obtained by the data acquisition module and extracts environmental feature values ​​and user behavior feature values;

[0014] Visual information generation module: Based on the feature value output of the data processing module, it dynamically generates visual communication content in combination with preset rules and algorithms;

[0015] Information transmission module: transmits the generated visual communication content to the display module through wireless communication protocols (such as Wi-Fi, 5G, LoRa);

[0016] Display module: used to display visual communication content in real time and interact with users, supporting multi-terminal display forms;

[0017] Feedback module: collects user interaction feedback information, such as behavior adjustment, direct rating, voice command, etc., and sends the feedback information back to the data processing module to update the model and optimize the communication content.

[0018] As a preferred solution of the present invention, the data processing module calculates the environmental characteristic value E by the following formula: t :

[0019]

[0020] Where: E t : Environmental characteristic value, reflecting the comprehensive state of the current environment; W ei : Weight coefficient of environmental sensor i, used to adjust the importance of different sensors; S ei The measurement values ​​of environmental sensor i, such as temperature and humidity;

[0021] n: total number of environmental sensors. Weight coefficient W ei Machine learning methods can be used to dynamically adjust based on historical data and scenario requirements to adapt to different application scenarios.

[0022] As a preferred solution of the present invention, the user behavior characteristic value U t The calculation formula is as follows:

[0023]

[0024] Among them: Among them: U t User behavior characteristic value, reflecting the comprehensive attention level of the user in the scene; B j The score of the jth behavior feature, such as gaze direction score, dwell time weight, etc.

[0025] T j : duration of the jth behavior; m: number of user behavior features. The calculation of user behavior feature values ​​combines the weight and duration of the behavior, which helps to dynamically evaluate the user's interest and engagement in the current content.

[0026] As a preferred solution of the present invention, the visual information generation module calculates the visual content priority P by the following formula: t :

[0027] P t =α·E t +β·U t

[0028] Where: P t : Visual content priority, used to determine the importance of displayed content; Et : Environmental characteristic value, U t : user behavior feature value, α and β: weight coefficients, which reflect the relative importance of environmental features and user behavior features respectively, and satisfy α + β = 1. The priority calculation formula combines the comprehensive influence of the environment and user behavior to dynamically adjust the order of visual communication content.

[0029] As a preferred solution of the present invention, the visual information generation module selects display content according to the following optimal decision formula:

[0030]

[0031] Where: C t : The final visual content of the presentation; Candidate content set; R c : The relevance score of candidate content C; P t : Priority. Through this formula, the system can dynamically select the content that best matches the current scene from the candidate content for display.

[0032] As a preferred solution of the present invention, after the feedback module collects user feedback, the relevance score R′ of the candidate content is dynamically updated by the following formula: c :

[0033] R c ′=R c +γ·ΔR

[0034] Where: R′ c : Updated relevance score; R c : Current relevance score; ΔR: The amount of adjustment to the content caused by user feedback; γ: Feedback adjustment weight, which reflects the influence of user feedback on the score update.

[0035] The dynamic collection and processing of user feedback enables the system to continuously optimize the content display logic.

[0036] As a preferred solution of the present invention, the adjustment amount ΔR fed back by the user is calculated by the following formula:

[0037]

[0038] Among them: F k : The actual feedback value of the kth user; The feedback value predicted by the system; p: number of feedback samples. The quantification of user feedback errors helps to dynamically adjust the content relevance score, thereby improving the accuracy of content matching.

[0039] As a preferred solution of the present invention, the data processing module normalizes the original data to eliminate the dimensional influence of data of different dimensions. The normalization formula is as follows:

[0040]

[0041] Where: S′ ei : Normalized data value; S min and S max : are the minimum and maximum values ​​of the sensor data respectively; S ei : Raw data value. Normalization ensures that the sensor data has a uniform scale for subsequent calculation of environmental feature values.

[0042] As a preferred solution of the present invention, the display module adopts augmented reality technology and adjusts the content transparency τ in real time according to the following formula t :

[0043]

[0044] Where: τ t : Content transparency, used for augmented reality display; P t : content priority; λ: adjustment coefficient; θ: transparency threshold, used to balance the prominence of content display. Dynamic adjustment of transparency helps optimize user experience and highlight key information.

[0045] The present invention also provides a visual communication information processing method based on the Internet of Things, comprising the following steps: obtaining environmental data S through a data acquisition module ei and user behavior data B j ; Use the data processing module to calculate the environmental characteristic value E t and user behavior characteristic value U t According to the formula P t =α·E t +β·U t Calculate content priority; use visual information generation module to select the best content C t And dynamically generate visual communication content; send the content to the display module through the information transmission module; the display module displays the content and collects user feedback information according to the feedback module k , adjust the content relevance score R′ c .

[0046] Beneficial effects:

[0047] The present invention collects environmental data (such as temperature and humidity, light intensity, crowd density, etc.) and user behavior data (such as gaze direction, dwell time, gesture operation, etc.) in real time through IoT sensors, and processes and analyzes them through intelligent algorithms to dynamically generate visual communication content that meets the needs of the current scene. Compared with the existing technology, the present invention solves the problem of static content, realizes seamless adaptation of visual information to the environment and user behavior, and greatly improves the efficiency of information transmission and the user's concentration.

[0048] The present invention calculates environmental feature values ​​and user behavior feature values, combines them with a content priority decision model, and intelligently selects the best content from candidate content for display, significantly improving the accuracy of visual communication. At the same time, the feedback module collects user interaction data and dynamically adjusts the content generation strategy, enabling the system to perform personalized optimization based on user interests, improving user participation and satisfaction, and solving the problem of extensive content selection in the prior art.

[0049] The present invention has designed a perfect feedback collection and processing mechanism, which can dynamically adjust content scoring and algorithm parameters through real-time feedback of user behavior data, forming a closed-loop optimization process. This adaptive optimization mechanism enables the system to continuously learn and iterate, and continuously improve the effect of content generation and display. Compared with the static or manual adjustment methods in the prior art, it greatly enhances the system's intelligence level and long-term performance stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 This is a system block diagram of a visual communication information processing system based on the Internet of Things. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The following is combined with Figure 1 With reference to the accompanying drawings and a number of embodiments, specific embodiments of the present invention are described in detail.

[0054] A visual communication information processing system based on the Internet of Things, including the following modules:

[0055] 1. Data acquisition module: obtains environmental data and user behavior data through the IoT sensor network. The environmental data includes temperature, humidity, light intensity, and sound physical parameters. The user behavior data includes gaze direction, dwell time, and gestures.

[0056] 2. Data processing module: Process the raw data obtained by the data acquisition module to extract environmental characteristic values ​​and user behavior characteristic values; the data processing module calculates the environmental characteristic value E through the following formula t :

[0057]

[0058] Where: E t : Environmental characteristic value, reflecting the comprehensive state of the current environment; W ei : Weight coefficient of environmental sensor i, used to adjust the importance of different sensors; S ei The measurement value of environmental sensor i; n: the total number of environmental sensors.

[0059] User behavior characteristic value U t The calculation formula is as follows:

[0060]

[0061] Among them: Among them: U t User behavior characteristic value, reflecting the comprehensive attention level of the user in the scene; B j The score of the jth behavior feature; T j : duration of the jth behavior; m: number of user behavior features.

[0062] The data processing module normalizes the original data to eliminate the dimensional influence of data of different dimensions. The normalization formula is as follows:

[0063]

[0064] Where: S′ ei : Normalized data value; S min and S max : are the minimum and maximum values ​​of the sensor data respectively; S ei : Original data value.

[0065] 3. Visual information generation module: According to the characteristic value output of the data processing module, combined with preset rules and algorithms, the visual communication content is dynamically generated; the visual information generation module calculates the visual content priority P by the following formula t :

[0066] P t =α·E t +β·U t

[0067] Where: P t : Visual content priority, used to determine the importance of displayed content; E t : Environmental characteristic value, Ut : user behavior feature value, α and β: weight coefficients, which reflect the relative importance of environmental features and user behavior features respectively, and satisfy α+β=1.

[0068] The visual information generation module selects display content according to the following optimal decision formula:

[0069]

[0070] Where: C t : The final visual content of the presentation; Candidate content set; R c : The relevance score of candidate content C; P t :priority.

[0071] 4. Information transmission module: transmits the generated visual communication content to the display module through the wireless communication protocol;

[0072] 5. Display module: used to display visual communication content in real time and interact with users, supporting multi-terminal display forms; the display module uses augmented reality technology and adjusts the content transparency in real time according to the following formula τ t :

[0073]

[0074] Where: τ t : Content transparency, used for augmented reality display; P t : content priority; λ: adjustment coefficient; θ: transparency threshold, used to balance the prominence of content display.

[0075] 6. Feedback module: collects user interactive feedback information, such as behavior adjustment, direct rating, voice command, etc., and sends the feedback information back to the data processing module to update the model and optimize the content. After collecting user feedback, the feedback module dynamically updates the relevance score R′ of the candidate content through the following formula c :

[0076] R c ′=R c +γ·ΔR

[0077] Where: R′ c : Updated relevance score; R c : Current relevance score; ΔR: The amount of adjustment to the content caused by user feedback; γ: Feedback adjustment weight, which reflects the influence of user feedback on the score update.

[0078] The adjustment amount ΔR from user feedback is calculated using the following formula:

[0079]

[0080] Among them: F k : The actual feedback value of the kth user; The feedback value predicted by the system; p: the number of feedback samples.

[0081] Implementation method 1: Data collection: In outdoor advertising scenarios, the system collects environmental data and user behavior data in real time through IoT devices, and dynamically generates and displays advertising content that best meets the needs of the scenario.

[0082] The data acquisition module collects the following information through the sensor network: Environmental data: Temperature S e1 =35℃, light intensity S e2 =800Ix, crowd density S e3 =50 people / m2;

[0083] User behavior data: The time the user stares at the screen is B1=8 seconds, and the interaction behavior score is B2=5.

[0084] Calculation of data processing environment characteristic values:

[0085] Take the weight coefficient W e 1=0.4、W e2 =0.3, W e 3=0.3:

[0086] E t =0.4·35+0.3·800+0.3·50=320

[0087] Calculation of user behavior feature value:

[0088]

[0089] Assume that the behavior weights are T1=0.6 and T2=0.4:

[0090]

[0091] Visual information generation: According to the priority formula: P t =α·E t +β·U t

[0092] Take weights α = 0.8, β = 0.2: P t =0.8·320+0.2·6.8=257.36

[0093] Content selection: According to the formula:

[0094] Candidate content collection The content relevance scores in are R c1=0.9, R c2 =0.6, R c3 =0.7.

[0095] Calculate the priority value: R c1 ·P t =0.9·257.36=231.62; R c2 ·P t =0.6·257.36=154.42; R c3 ·P t =0.7·257.36=180.15; select C1 as the display content.

[0096] Content display: The display module plays C1 (cold drink advertisement) and dynamically adjusts the content transparency. The transparency formula is:

[0097] Take λ=0.5, θ=200,

[0098]

[0099] Set the transparency to 87%.

[0100] Feedback adjustment: Collect user click behavior and dwell time, and dynamically update the content relevance score through the formula:

[0101] R c ′=R c +γ·ΔR

[0102] Assume that the user's feedback score adjustment for C1 is ΔR = 0.1 and the adjustment coefficient γ = 0.8:

[0103] R′ c1 =0.9+0.8·0.1=0.98

[0104] Implementation 2: Data collection environment data: exhibition hall temperature S e1 =22℃, light intensity S e2 =400lx;

[0105] User behavior data: gaze direction B1 = 10 seconds, voice interaction frequency B2 = 3.

[0106] Data processing environment characteristic value E t =0.6·22+0.4·400=168.8; User behavior characteristic value

[0107] Visual information generation priority P t =0.5·168.8+0.5·7.9=88.35.

[0108] The candidate contents are the history of the exhibits and interactive experience videos, and the content with the highest relevance score is selected.

[0109] Content display: The display screen dynamically plays the history of exhibits, and the transparency is adjusted to 95%. Feedback adjustment If the user continues to stay, the system will increase the current content score.

[0110] Implementation 3: Data collection: shopping mall area temperature S e1 =25℃, flow density S e2 = 60 people; User behavior: The user’s current location B1 is 10 meters away from the target store, and the time spent looking at the discount information is B2 = 15 seconds.

[0111] Data processing: E t =0.5·25+0.5·60=42.5,

[0112] Visual Information Generation: P t =0.7·42.5+0.3·12=34.55.

[0113] Content display: Display path and target store discount information, with transparency τt=90%.

[0114] Feedback adjustment: When users enter the target store, their behaviors are recorded to optimize navigation strategies.

[0115] Implementation 4: Data collection: Environmental data: Temperature S e1 =18℃, humidity S e2 =70%; User behavior: gaze time at the station B1 = 5 seconds, total stay time B2 = 30 seconds.

[0116] Data Processing t =0.4·18+0.6·70=48.2,

[0117] Visual Information Generation t =0.6·48.2+0.4·22.5=38.18.

[0118] Content display shows the next bus information, transparency τ t =85%.

[0119] Feedback adjustment re-adjusts the order of content when user attention decreases.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A visual communication information processing system based on the Internet of Things, characterized in that: Includes the following modules: Data collection module: obtains environmental data and user behavior data through the IoT sensor network. The environmental data includes temperature, humidity, light intensity, and sound physical parameters. The user behavior data includes gaze direction, dwell time, and gestures. Data processing module: processes the raw data obtained by the data acquisition module and extracts environmental feature values ​​and user behavior feature values; Visual information generation module: Based on the feature value output of the data processing module, it dynamically generates visual communication content in combination with preset rules and algorithms; Information transmission module: transmits the generated visual communication content to the display module through wireless communication protocol; Display module: used to display visual communication content in real time and interact with users, supporting multi-terminal display forms; Feedback module: collects user interaction feedback information, such as behavior adjustment, direct rating, and voice command, and sends the feedback information back to the data processing module to update the model and optimize the communication content.

2. According to claim 1, a visual communication information processing system based on the Internet of Things is characterized in that: The data processing module calculates the environmental characteristic value E by the following formula t : Where: E t : Environmental characteristic value, reflecting the comprehensive state of the current environment; W ei : Weight coefficient of environmental sensor i, used to adjust the importance of different sensors; S ei The measurement value of environmental sensor i; n: the total number of environmental sensors.

3. According to the visual communication information processing system based on the Internet of Things in claim 1, it is characterized in that: The user behavior characteristic value U t The calculation formula is as follows: Among them: Among them: U t User behavior characteristic value, reflecting the comprehensive attention level of the user in the scene; B j The score of the jth behavior feature; T j : duration of the jth behavior; m: number of user behavior features.

4. The visual communication information processing system based on the Internet of Things according to claim 1 is characterized in that: The visual information generation module calculates the visual content priority P by the following formula t : P t =α·E t +β·U t Where: P t : Visual content priority, used to determine the importance of displayed content; E t : Environmental characteristic value, U t : user behavior feature value, α and β: weight coefficients, which reflect the relative importance of environmental features and user behavior features respectively, and satisfy α+β=1.

5. The visual communication information processing system based on the Internet of Things according to claim 4 is characterized in that: The visual information generation module selects display content according to the following optimal decision formula: Where: C t : The final visual content of the presentation; Candidate content set; R c : The relevance score of candidate content C; P t : Priority, combined with the result calculated in claim 4.

6. The visual communication information processing system based on the Internet of Things according to claim 1 is characterized in that: After the feedback module collects user feedback, it dynamically updates the relevance score R′ of the candidate content through the following formula: c : R c ′=R c +γ·ΔR Where: R′ c : Updated relevance score; R c : Current relevance score; ΔR: The amount of adjustment to the content caused by user feedback; γ: Feedback adjustment weight, which reflects the influence of user feedback on the score update.

7. The visual communication information processing system based on the Internet of Things according to claim 6 is characterized in that: The adjustment amount ΔR fed back by the user is calculated by the following formula: Among them: F k : The actual feedback value of the kth user; The feedback value predicted by the system; p: the number of feedback samples.

8. The visual communication information processing system based on the Internet of Things according to claim 1 is characterized in that: The data processing module normalizes the original data to eliminate the dimensional influence of data of different dimensions. The normalization formula is as follows: Where: S′ ei : Normalized data value; S min and S max : are the minimum and maximum values ​​of the sensor data respectively; S ei : Original data value.

9. The visual communication information processing system based on the Internet of Things according to claim 1 is characterized in that: The display module uses augmented reality technology and adjusts the content transparency τ in real time according to the following formula t : Where: τ t : Content transparency, used for augmented reality display; P t : Content priority; λ: Adjustment coefficient; θ: Transparency threshold, used to balance the prominence of content display.

10. A visual communication information processing method based on the Internet of Things according to any one of claims 1 to 9, characterized in that: The following steps are involved: Acquire environmental data S through the data acquisition module ei and user behavior data B j ; Use the data processing module to calculate the environmental characteristic value E t and user behavior characteristic value U t ; According to the formula P t =α·E t +β·U t Calculate content priority; Use the visual information generation module to select the optimal content C t and dynamically generates visual communication content; sends the content to the display module through the information transmission module; The display module displays content and collects user feedback information based on the feedback module F k , adjust the content relevance score R′ c .

Images