Product data intelligent analysis system and method based on Internet of Things

Abstract

The invention discloses an intelligent product data analysis system and method based on the Internet of Things, and relates to the technical field of big data analysis, an image sensor is used for collecting perception data including the number of watching people, time, face images and the like, a reference graph and a datum line are constructed according to coordinates of the left eye, the right eye and the nose, the face watching state is accurately judged, and the user experience is improved. And the reliability and the practicability of the data are greatly enhanced. According to the method, the number of watching people and watching time are comprehensively considered, and the watching effect score of the display information is calculated through analysis, so that the display resources are reasonably distributed, the attention degree of the display information is effectively quantified, the information display efficiency is improved, the user interaction experience is optimized, and the released information better meets the requirements and expectation of the user. A swivel deviation threshold is determined in combination with neural network model training, flexible adjustment is performed in combination with confidence, and a swivel fixation threshold is set to screen effective fixation, so that the scheme is more rigorous and comprehensive in data processing and analysis, and the delivery quality and benefits are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of big data analysis, and in particular to a product data intelligent analysis system and method based on the Internet of Things. Background Art

[0002] Billboards are important communication media in urban environments, with extensive information transmission and brand promotion functions. Through eye-catching locations and diverse content display, they provide companies with a channel to directly reach target audiences, while also enriching the city's business atmosphere. Modern billboards not only assume the traditional propaganda role, but also integrate intelligent technology to improve advertising effects and audience experience through real-time data analysis, precise delivery and dynamic content switching. In addition, billboards can also add cultural value to the city, such as public welfare propaganda or art design, and become an important link between the city and the public.

[0003] When billboards promote advertisements, it is difficult to accurately judge the actual level of attention paid to the content being played due to the lack of real-time monitoring of the crowd's attention. This makes it impossible to quantify the advertising effect and adjust the content according to the interests of the crowd, which affects the targeting and efficiency of advertising. To solve this problem, it is necessary to capture the crowd's sight and behavior data, so as to dynamically optimize the advertising content and improve the substantiveness and accuracy of the promotion. Summary of the invention

[0004] The purpose of the present invention is to provide a product data intelligent analysis system and method based on the Internet of Things to solve the problems raised in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solution: a product data intelligent analysis method based on the Internet of Things, the intelligent analysis method comprising the following steps: Step S1, setting a data collection cycle, obtaining the perception data of the information display terminal in each data collection cycle through the sensor, the perception data including the number of gazers, gaze time and face image; Step S1-1, set the data collection period T. When using the data collection period T, mark the corresponding timestamp, specifically, the time of the data collection period T corresponds to an actual timestamp in real time; Step S1-2, obtaining the perception data in the information display area of ​​the information display terminal through the image sensor within the data collection period T, wherein the information display area of ​​the information display terminal is represented as an area that can be contacted by the user's line of sight and can view the content displayed by the information display terminal, and the perception data includes the number of gazers, the gaze time and the face image, and the gaze time is represented as the time from the start of gaze to the end of gaze of a gaze unit gazing at the information display terminal.

[0006] The number of gazers refers to the number of users within the effective gaze angle range within a specific time period. The effective gaze angle refers to the range in which the user's eyes are directed toward the information display terminal and remain there for a certain period of time. The number of people who meet this condition is the number of gazers; Step S2, extracting a face image from the perception data for analysis, wherein the face image includes a left eye two-dimensional coordinate, a right eye two-dimensional coordinate, and a nose two-dimensional coordinate, wherein each face image corresponds to a basic two-dimensional coordinate, and a reference graph is constructed by connecting the left eye two-dimensional coordinate, the right eye two-dimensional coordinate, and the nose two-dimensional coordinate; Step S2-1, extracting a face image according to the perception data collected by the image sensor, wherein the face image includes a left eye two-dimensional coordinate, a right eye two-dimensional coordinate and a nose two-dimensional coordinate, and one face image corresponds to one basic two-dimensional coordinate, which is specifically expressed as constructing an independent two-dimensional coordinate for each face image, wherein the abscissa represents the horizontal position in the face image, and the ordinate represents the vertical position in the face image, and the nose two-dimensional coordinate is taken as the origin coordinate, and the left eye two-dimensional coordinate is obtained as (x L ,y L ), the two-dimensional coordinates of the right eye are (x R ,y R ), the two-dimensional coordinates of the nose are (x0, y0); Step S2-2, by connecting the two-dimensional coordinates of the left eye and the two-dimensional coordinates of the right eye, the two-dimensional coordinates of the left eye and the two-dimensional coordinates of the nose, and the two-dimensional coordinates of the right eye and the two-dimensional coordinates of the nose, a geometric structure consisting of three line segments is formed as a reference figure for facial features.

[0007] By extracting the position coordinates of the left eye, right eye and nose, and using these coordinates to establish connecting lines to form a geometric structure, the features of the face can be accurately captured.

[0008] Step S3, judging whether the face is looking at the information display terminal according to the reference pattern, constructing a horizontal reference line of the face using the connection line of the left eye two-dimensional coordinates and the right eye two-dimensional coordinates, and using the two-dimensional coordinates of the nose as a perpendicular line to the horizontal reference line of the face, and then constructing a vertical reference line of the face, and judging whether the vision of the current face image is a valid gaze angle in combination with the horizontal reference line of the face; Step S3-1, construct the horizontal reference line of the face using the line connecting the two-dimensional coordinates of the left eye and the two-dimensional coordinates of the right eye, use the two-dimensional coordinates of the nose as the perpendicular line to the horizontal reference line of the face, and then construct the vertical reference line of the face, and record the perpendicular point as the vertical reference point of the face, the two-dimensional coordinates are (x N ,y N ); Step S3-2, setting the head turning deviation threshold, inputting multiple groups of face image data into the neural network model for training, obtaining the basic threshold Ebase for head turning error judgment, and calculating and setting the head turning deviation threshold according to the basic threshold Ebase. The following formula is used to calculate and set the head turning deviation threshold: ; In the formula, E adj It is represented as the head turning deviation threshold; Ebase is represented as the basic threshold of head turning error judgment obtained by training the neural network model; m is represented as the adjustment coefficient, which is the sensitivity of confidence to the head turning deviation threshold; C is represented as confidence; the range of C is 0≤C≤1, which represents the confidence in the face-looking judgment. When C=1, it means that there is complete confidence and the basic threshold of head turning error judgment is accurate. When C=0, it means that there is no confidence and the basic threshold of head turning error judgment needs to be further refined.

[0009] Step S3-3, judging whether the face image is facing the information display terminal according to the two-dimensional coordinates of the facial vertical reference point, respectively calculating the distance from the facial vertical reference point to the left eye and the distance to the right eye, and calculating according to the Euclidean distance formula, which is as follows: ; Where D represents the Euclidean distance between two points, that is, the calculated straight-line distance; x1, y1 represent the two-dimensional coordinates of the first point, and the two-dimensional coordinates are specifically (x1, y1); x2, y2 represent the two-dimensional coordinates of the second point, and the two-dimensional coordinates are specifically (x2, y2); The left eye 2D coordinate (x L ,y L ) and the facial vertical reference point (x N ,y N ) Substitute into the Euclidean distance formula to get the straight-line distance D from the vertical reference point of the face to the left eye L ; The right eye 2D coordinate (x R ,y R ) and the facial vertical reference point (x N ,y N ) Substitute into the Euclidean distance formula to get the straight-line distance D from the vertical reference point of the face to the left eye R ; The vertical reference point of the face (x N ,y N ) and the two-dimensional coordinates of the nose (0, 0) are substituted into the Euclidean distance formula to obtain the straight-line distance D0 from the vertical reference point of the face to the nose; The left eye 2D coordinate (x L ,y L ) and the right eye 2D coordinate (x R ,yR ) Substitute into the Euclidean distance formula to obtain the straight-line distance D from the left eye to the right eye L-R , and extract the maximum distance D from the left eye to the right eye within the current data acquisition period T max , as well as the maximum distance D from the nose to the facial vertical reference point (x N , y N ); max-0 ; Step S3-4: Construct a front-view two-dimensional coordinate system with the geometric center of the information display terminal as the origin, where the abscissa represents the horizontal direction and the ordinate represents the vertical direction. Map the face image in the front-view two-dimensional coordinate system, divide the face image with abscissa 0 into the front-view type, and divide the face image with non-zero abscissa into the non-front-view type; Step S3-4-1: In the front-view type, judge the head turn according to the maximum distance D max from the left eye to the right eye and the actually obtained and calculated straight-line distance D L-R ; When D max = D L-R , judge that the current face image is a front-view information display terminal; When D max > D L-R , judge that the current face image is a non-front-view information display terminal; Step S3-4-2: In the non-front-view type, judge the head turn according to the straight-line distance D L from the facial vertical reference point to the left eye and the straight-line distance D R from the facial vertical reference point to the left eye; When |D L - D R | ≤ E adj , judge that the gaze angle of the face image is a front-view information display terminal; When |D L - D R | > E adj , judge that the gaze angle of the face image is a non-front-view information display terminal.

[0010] Step S3-4-2: In the non-front-view type, judge the head turn according to the straight-line distance D L from the facial vertical reference point to the left eye and the straight-line distance D R from the facial vertical reference point to the left eye; When |D L - D R | ≤ E adj , judge that the gaze angle of the face image is a front-view information display terminal; When |D L - D R丨>E adj When it is, it is determined that the gaze angle of the face image is a non-frontal view of the information display terminal.

[0011] Step S3-5: Based on the initial two-dimensional coordinates (0, 0) of the nose, combined with the vertically coordinated nose obtained in real time, determine whether the face image is looking up or down; When D0 = D max-0 it is determined that the initial face image is facing the information display terminal frontally. When the vertically coordinated nose rises, it is determined that the face image is looking up. When the vertically coordinated nose drops, it is determined that the face image is looking down; When D0 < D max-0 extract the vertically coordinated value when calculating D max-0 ; When D max-0 has a negative vertically coordinated value, it is determined that the initial face image is looking up; When D max-0 has a positive vertically coordinated value, it is determined that the initial face image is looking down; Step S3-6: Set the turning gaze threshold Q max , calculate the viewing angle after turning according to the data in the non-frontal view type, and filter out the effective gaze. The calculation is as follows: ; In the formula, Q avg represents the viewing angle between the face image and the information display terminal; When Q avg ≤Q max it is determined that the face image has an effective gaze after turning; When Q avg >Q max it is determined that the face image has an ineffective gaze after turning.

[0012] By analyzing the eye and nose coordinates in the face image, constructing a reference line and calculating the Euclidean distance, it is determined whether the face is facing the information display terminal frontally. Combining the turning deviation threshold obtained from neural network training and geometric calculations, accurately evaluate the gaze angle, and determine whether it is an effective gaze or a turn to ensure the accuracy of interaction.

[0013] Step S4: Calculate the number of people with effective gazes within the data collection period, and calculate the total effective gaze time within the data collection period according to the gaze time corresponding to the number of people with gazes. Combining the time stamp corresponding to the data collection period, conduct data statistics on the information displayed in the information display terminal. The data statistics include the total number of people with gazes and the total gaze time corresponding to the information; Step S4-1: Set the gaze time threshold S max , and among the effective gazes of the face image, filter out those with a gaze time greater than the gaze time threshold Smax The number of people looking at the face, and the corresponding looking time of each face image; Step S4-2: Perform statistical analysis on the content displayed in the information display terminal according to the actual timestamp corresponding to the data collection period T, wherein the number of gazers refers to the number of people who are within the effective gaze range and whose gaze time exceeds the set threshold S max The total fixation time refers to the cumulative total fixation time of face images that meet the effective fixation range and fixation time conditions.

[0014] By screening out the number of gazers that meet the effective gaze range and gaze time threshold, and counting their gaze time, the interactive effect of the information display terminal can be accurately measured. Combined with timestamps, detailed gaze data statistics are provided to help optimize display content and user experience.

[0015] Step S5: evaluating the gaze effect of each display information according to the number of gazers that meet the effective gaze angle and the total gaze time during the data collection period.

[0016] The gaze effect of each displayed information is calculated using the number of annotations in the data collection period T and the total gaze time. The gaze effect is calculated using the following formula: ; Where P i It is represented by the gaze effect score of the i-th displayed information on the information display terminal; w1 is the weight of the number of gazers; w2 is the weight of the gaze time; N i It is represented by the actual number of people watching the i-th displayed information; t i It is represented as the total fixation time of the i-th displayed information; N max It is expressed as the maximum value of the number of historical viewers; t max It is expressed as the maximum value of the sum of historical fixation time; Display resources are allocated to the display information of the information display terminal according to the gaze effect calculation result.

[0017] By combining the number of gazers and gaze time, the gaze effect of displayed information is quantified, and resource allocation of displayed content is optimized. The calculation of gaze effect score ensures that resources are allocated first to the most concerned information, improving the efficiency of information display and user interaction experience.

[0018] Furthermore, a product data intelligent analysis system based on the Internet of Things, the intelligent analysis system includes a data acquisition module, a face image analysis module, a gaze angle judgment module, a gaze statistics module and a gaze effect evaluation resource allocation module; The data acquisition module is used to regularly collect user perception data, including the number of gazers, gaze time and face images; the face image analysis module is used to extract the user's facial image data, analyze the coordinates of the eyes and nose, and construct a reference graph; the gaze angle judgment module is used to judge whether the gaze is at the information display terminal and whether it meets the effective gaze angle according to the face image analysis result; the gaze statistics module is used to count the number of gazers and the total gaze time within the effective gaze range; the gaze effect evaluation resource allocation module is used to evaluate the gaze effect of the display content according to the statistical data and reasonably allocate display resources; The output end of the data acquisition module is electrically connected to the input end of the face image analysis module; the output end of the face image analysis module is electrically connected to the input end of the gaze angle judgment module; the output end of the gaze angle judgment module is electrically connected to the input end of the gaze statistics module; the output end of the gaze statistics module is electrically connected to the input end of the gaze effect evaluation resource allocation module.

[0019] The data acquisition module includes a data acquisition cycle setting unit and a perception data acquisition unit; the data acquisition cycle setting unit is used to set and adjust the data acquisition cycle T; the perception data acquisition unit is used to acquire the perception data in the information display area of ​​the information display terminal through the image sensor within a specified period; The face image analysis module includes a face image extraction unit and a reference graph construction unit; the face image extraction unit is used to extract the face image of each user from the perception data, and obtain the coordinates of the left eye, the right eye and the nose in the two-dimensional coordinate system; the reference graph construction unit is used to construct a reference image; The gaze angle judgment module includes a reference line construction unit and a gaze angle judgment unit; the reference line construction unit is used to construct a horizontal reference line of the face and a vertical reference line of the face; the gaze angle judgment unit is used to judge whether the face is looking at the information display terminal; The gaze statistics module includes a gaze number statistics unit and a gaze time statistics unit; the effective gaze range statistics unit is used to count the number of annotations; the gaze time statistics unit is used to count the gaze time; The gaze effect evaluation resource allocation module includes a gaze effect evaluation unit and a display resource allocation unit; the gaze effect evaluation unit is used to calculate the gaze effect of each display information according to the number of gazers and the gaze time; the display resource allocation unit is used to reasonably allocate the display resources of the information display terminal according to the gaze effect score.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention can effectively capture the sight and behavior data of the crowd through accurate perception data collection and analysis. By using image sensors to obtain information including the number of people looking at, time and face images, and using the coordinates of the left eye, right eye and nose to build reference graphics and baselines to judge the gaze state, it can accurately judge in real time whether the face is looking at the information display terminal, thereby providing an accurate attention data basis for advertising delivery and improving the reliability and practicality of the data.

[0021] 2. The present invention comprehensively considers the number of people watching and the watching time, calculates the watching effect score of each displayed information through a specific formula, and allocates display resources based on the score. This method can scientifically quantify the degree of attention paid to the displayed information, ensure that resources are preferentially allocated to information with high attention, effectively improve the efficiency of information display, optimize the user interaction experience, make advertising delivery more targeted, and better meet user needs and advertiser expectations.

[0022] 3. The present invention enhances the accuracy and flexibility of judgment by using neural network model training to determine the head turning deviation threshold. The threshold is adjusted in combination with factors such as confidence to adapt to different scenarios and individual differences. At the same time, by setting the head turning gaze threshold to screen effective gaze, the gaze situation under non-emmetropia type is further accurately analyzed. The overall technical solution is rigorous and comprehensive in data processing and analysis, which effectively overcomes the problem of lack of real-time monitoring of crowd attention and difficulty in quantifying the effect of traditional advertising, and improves the overall quality and efficiency of advertising. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a method for intelligent product data analysis based on the Internet of Things according to the present invention; Figure 2 This is a structural schematic diagram of a product data intelligent analysis system based on the Internet of Things of the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] Embodiment 1: Figure 1 As shown, the present invention provides a technical solution, a product data intelligent analysis method based on the Internet of Things, and the intelligent analysis method comprises the following steps: Step S1, setting a data collection cycle, obtaining the perception data of the information display terminal in each data collection cycle through the sensor, the perception data including the number of gazers, gaze time and face image; Step S1-1, set the data collection period T. When using the data collection period T, mark the corresponding timestamp, specifically, the time of the data collection period T corresponds to an actual timestamp in real time; Step S1-2, obtaining the perception data in the information display area of ​​the information display terminal through the image sensor within the data collection period T, wherein the information display area of ​​the information display terminal is represented as an area that can be contacted by the user's line of sight and can view the content displayed by the information display terminal, and the perception data includes the number of gazers, the gaze time and the face image, and the gaze time is represented as the time from the start of gaze to the end of gaze of a gaze unit gazing at the information display terminal.

[0026] In the specific implementation, taking the advertisement displayed on the billboard of a shopping mall as an example, when people enter the area where their eyes are in contact with and view the content displayed by the information display terminal, the face data of the people in the area is obtained through the high-definition camera; Step S2, extracting a face image from the perception data for analysis, wherein the face image includes a left eye two-dimensional coordinate, a right eye two-dimensional coordinate, and a nose two-dimensional coordinate, wherein each face image corresponds to a basic two-dimensional coordinate, and a reference graph is constructed by connecting the left eye two-dimensional coordinate, the right eye two-dimensional coordinate, and the nose two-dimensional coordinate; Step S2-1, extracting a face image according to the perception data collected by the image sensor, wherein the face image includes a left eye two-dimensional coordinate, a right eye two-dimensional coordinate and a nose two-dimensional coordinate, and one face image corresponds to one basic two-dimensional coordinate, which is specifically expressed as constructing an independent two-dimensional coordinate for each face image, wherein the abscissa represents the horizontal position in the face image, and the ordinate represents the vertical position in the face image, and the nose two-dimensional coordinate is taken as the origin coordinate, and the left eye two-dimensional coordinate is obtained as (x L ,y L ), the two-dimensional coordinates of the right eye are (x R ,y R ), the two-dimensional coordinates of the nose are (x0, y0); Step S2-2, by connecting the two-dimensional coordinates of the left eye and the two-dimensional coordinates of the right eye, the two-dimensional coordinates of the left eye and the two-dimensional coordinates of the nose, and the two-dimensional coordinates of the right eye and the two-dimensional coordinates of the nose, a geometric structure consisting of three line segments is formed as a reference figure for facial features.

[0027] During the above-mentioned two-dimensional coordinate system construction process, a separate two-dimensional coordinate system is constructed for each face image, and the two-dimensional coordinate system is constructed with the nose in the face image data obtained for the first time as the origin. The horizontal axis represents the horizontal position in the face image, and the vertical axis represents the vertical position in the face image. The corresponding coordinate data of the left eye and the right eye in the two-dimensional coordinate system are extracted.

[0028] Step S3, judging whether the face is looking at the information display terminal according to the reference pattern, constructing a horizontal reference line of the face using the connection line of the left eye two-dimensional coordinates and the right eye two-dimensional coordinates, and using the two-dimensional coordinates of the nose as a perpendicular line to the horizontal reference line of the face, and then constructing a vertical reference line of the face, and judging whether the vision of the current face image is a valid gaze angle in combination with the horizontal reference line of the face; Step S3-1, construct the horizontal reference line of the face using the line connecting the two-dimensional coordinates of the left eye and the two-dimensional coordinates of the right eye, use the two-dimensional coordinates of the nose as the perpendicular line to the horizontal reference line of the face, and then construct the vertical reference line of the face, and record the perpendicular point as the vertical reference point of the face, the two-dimensional coordinates are (x N ,y N ); Step S3-2, setting the head turning deviation threshold, inputting multiple groups of face image data into the neural network model for training, obtaining the basic threshold Ebase for head turning error judgment, and calculating and setting the head turning deviation threshold according to the basic threshold Ebase. The following formula is used to calculate and set the head turning deviation threshold: ; In the formula, E adj It is represented as the head turning deviation threshold; Ebase is represented as the basic threshold of head turning error judgment obtained through neural network model training; m is represented as the adjustment coefficient, the sensitivity of confidence to the head turning deviation threshold; C is represented as confidence; Step S3-3, judging whether the face image is facing the information display terminal according to the two-dimensional coordinates of the facial vertical reference point, respectively calculating the distance from the facial vertical reference point to the left eye and the distance to the right eye, and calculating according to the Euclidean distance formula, which is as follows: ; Where D represents the Euclidean distance between two points, that is, the calculated straight-line distance; x1, y1 represent the two-dimensional coordinates of the first point, and the two-dimensional coordinates are specifically (x1, y1); x2, y2 represent the two-dimensional coordinates of the second point, and the two-dimensional coordinates are specifically (x2, y2); The left eye 2D coordinate (x L ,y L ) and the facial vertical reference point (x N ,y N ) Substitute into the Euclidean distance formula to get the straight-line distance D from the vertical reference point of the face to the left eye L ; Bring the two-dimensional coordinates of the right eye (x R , y R ) and the facial vertical reference point (x N , y N ) into the Euclidean distance formula to obtain the straight-line distance D from the facial vertical reference point to the left eye R ; Bring the facial vertical reference point (x N , y N ) and the two-dimensional coordinates of the nose (0, 0) into the Euclidean distance formula to obtain the straight-line distance D0 from the facial vertical reference point to the nose; Bring the two-dimensional coordinates of the left eye (x L , y L ) and the two-dimensional coordinates of the right eye (x R , y R ) into the Euclidean distance formula to obtain the straight-line distance D from the left eye to the right eye L-R , and extract the maximum distance D from the left eye to the right eye within the current data acquisition period T max , as well as the maximum distance D from the nose to the facial vertical reference point (x N , y N ) max-0 ; Step S3-4: Construct a front-facing two-dimensional coordinate system with the geometric center of the information display terminal as the origin, where the abscissa represents the horizontal direction and the ordinate represents the vertical direction. Map the face image in the front-facing two-dimensional coordinate system, divide the face image with abscissa 0 into the front-facing type, and divide the face image with non-zero abscissa into the non-front-facing type; Step S3-4-1: In the front-facing type, judge the head rotation according to the maximum distance D max from the left eye to the right eye and the actually obtained and calculated straight-line distance D L-R ; When D max = D L-R , judge that the current face image is facing the information display terminal; When D max > D L-R , judge that the current face image is not facing the information display terminal; Step S3-4-2: In the non-front-facing type, judge the head rotation according to the straight-line distance D L from the facial vertical reference point to the left eye and the straight-line distance D R from the facial vertical reference point to the left eye; When |D L - D R | ≤ E adj , judge that the viewing angle of the face image is facing the information display terminal; When |DL -D R 丨>E adj When it is, it is determined that the gaze angle of the face image is a non-frontal view of the information display terminal; Step S3-5: Based on the initial two-dimensional coordinates (0, 0) of the nose, combined with the nose ordinate obtained in real time, determine whether the face image is looking up or down; When D0 = D max-0 it is determined that the initial face image is facing the information display terminal head-on. When the ordinate of the nose rises, it is determined that the face image is looking up. When the ordinate of the nose drops, it is determined that the face image is looking down; When D0 < D max-0 at this time, extract and calculate D max-0 the ordinate at this time; When D max-0 the ordinate of is negative, it is determined that the initial face image is looking up; When D max-0 the ordinate of is positive, it is determined that the initial face image is looking down; Step S3-6: Set the turning gaze threshold Q max , calculate the viewing angle after turning according to the data in the non-frontal view type, and screen out the effective gaze. The calculation is as follows: ; In the formula, Q avg represents the viewing angle between the face image and the information display terminal; When Q avg ≤Q max it is determined that the face image has an effective gaze after turning; When Q avg >Q max it is determined that the face image has an ineffective gaze after turning.

[0029] For example, the left eye two-dimensional coordinates of a face image data in the information display area are (x L , y L ) = (200, 300), the right eye two-dimensional coordinates are (x R , y R ) = (400, 300), and the initial nose two-dimensional coordinates are (x0, y0) = (300, 400); Calculate the distance to the facial vertical reference point: From the left eye to the facial vertical reference point (nose): D L = ; From the right eye to the facial vertical reference point (nose): D R = ; The distance D0 from the nose to the vertical reference point on the face is: D0 = ; In the frontal view type, calculate the maximum distance D between the left eye and the right eye max = 200: D L-R = ; Since D max = D L-R , it is determined that the current face image is a frontal view information display terminal; Analyze another face image, and calculate the distance D from the left eye to the vertical reference point L = 140, the distance D from the right eye to the vertical reference point R = 160, and the head-turning deviation threshold E base = 5; Calculate |D L - D R | = |140 - 160| = 20 > 5; it is determined that the gaze angle of the face image is a non-frontal view information display terminal.

[0030] The ordinate of the nose is 400, and the calculated ordinate y0 of the facial vertical reference point is 500, and the calculated D0 = 500; When D0 = D max-0 , it indicates that the initial face image is a frontal view. When the ordinate of the nose rises, it is determined as looking up, and when the ordinate drops, it is determined as looking down.

[0031] Step S4, calculate the number of people with effective gaze within the data acquisition period, and calculate the total effective gaze time within the data acquisition period based on the gaze time corresponding to the number of people with gaze. Combine the time stamp corresponding to the data acquisition period to perform data statistics on the information displayed in the information display terminal. The data statistics include the number of people with gaze corresponding to the information and the total gaze time; Step S4-1, set the gaze time threshold S max , and in the effective gaze of the face image, filter out the number of people with a gaze time greater than the gaze time threshold S max , as well as the gaze time corresponding to each face image; Step S4-2, perform statistical analysis on the content displayed in the information display terminal according to the actual time stamp corresponding to the data acquisition period T. Among them, the number of people with gaze refers to the number of face images within the effective gaze range and with a gaze time exceeding the set threshold S max ; the total gaze time refers to the cumulative total of the gaze times of the face images that meet the conditions of the effective gaze range and the gaze time condition.

[0032] Step S5: evaluating the gaze effect of each display information according to the number of gazers that meet the effective gaze angle and the total gaze time during the data collection period.

[0033] The gaze effect of each displayed information is calculated using the number of annotations in the data collection period T and the total gaze time. The gaze effect is calculated using the following formula: ; Where P i It is represented by the gaze effect score of the i-th displayed information on the information display terminal; w1 is the weight of the number of gazers; w2 is the weight of the gaze time; N i It is represented by the actual number of people watching the i-th displayed information; t i It is represented as the total fixation time of the i-th displayed information; N max It is expressed as the maximum value of the number of historical viewers; t max It is expressed as the maximum value of the sum of historical fixation time; Display resources are allocated to the display information of the information display terminal according to the gaze effect calculation result.

[0034] For example, Display information 1: Number of gazers N1=50, total gaze time t1=200 seconds; Display information 2: Number of gazers N2=30, total gaze time t2=150 seconds; Display information 3: Number of gazers N3=70, total gaze time t3=300 seconds; The historical maximum number of viewers is N max =100, time t max =400 seconds; w1=0.5; w2=0.5; According to the formula, we can get: P1=(25 / 100)+(100 / 400)=0.25+0.25=0.5; P2=(15 / 100)+(75 / 400)=0.15+0.1875=0.3375; P3=(35 / 100)+(150 / 400)=0.35+0.375=0.725; The scoring result is P3>P1>P2; Embodiment 2, as Figure 2 As shown, the present invention provides a product data intelligent analysis system based on the Internet of Things, the intelligent analysis system includes a data acquisition module, a face image analysis module, a gaze angle judgment module, a gaze statistics module and a gaze effect evaluation resource allocation module; The data acquisition module is used to regularly collect user perception data, including the number of gazers, gaze time and face images; the face image analysis module is used to extract the user's facial image data, analyze the coordinates of the eyes and nose, and construct a reference graph; the gaze angle judgment module is used to judge whether the gaze is at the information display terminal and whether it meets the effective gaze angle according to the face image analysis result; the gaze statistics module is used to count the number of gazers and the total gaze time within the effective gaze range; the gaze effect evaluation resource allocation module is used to evaluate the gaze effect of the display content according to the statistical data and reasonably allocate display resources; The output end of the data acquisition module is electrically connected to the input end of the face image analysis module; the output end of the face image analysis module is electrically connected to the input end of the gaze angle judgment module; the output end of the gaze angle judgment module is electrically connected to the input end of the gaze statistics module; the output end of the gaze statistics module is electrically connected to the input end of the gaze effect evaluation resource allocation module.

[0035] The data acquisition module includes a data acquisition cycle setting unit and a perception data acquisition unit; the data acquisition cycle setting unit is used to set and adjust the data acquisition cycle T; the perception data acquisition unit is used to acquire the perception data in the information display area of ​​the information display terminal through the image sensor within a specified period; The face image analysis module includes a face image extraction unit and a reference graph construction unit; the face image extraction unit is used to extract the face image of each user from the perception data, and obtain the coordinates of the left eye, the right eye and the nose in the two-dimensional coordinate system; the reference graph construction unit is used to construct a reference image; The gaze angle judgment module includes a baseline construction unit and a gaze angle judgment unit; the baseline construction unit is used to construct a horizontal baseline of the face and a vertical baseline of the face; the gaze angle judgment unit is used to judge whether the face is looking at the information display terminal.

[0036] The gaze statistics module includes a gaze number statistics unit and a gaze time statistics unit; the effective gaze range statistics unit is used to count the number of annotations; the gaze time statistics unit is used to count the gaze time; The gaze effect evaluation resource allocation module includes a gaze effect evaluation unit and a display resource allocation unit; the gaze effect evaluation unit is used to calculate the gaze effect of each display information according to the number of gazers and the gaze time; the display resource allocation unit is used to reasonably allocate the display resources of the information display terminal according to the gaze effect score.

[0037] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A product data intelligent analysis method based on the Internet of Things, characterized by: The intelligent analysis method includes the following steps: Step S1, setting a data collection cycle, obtaining the perception data of the information display terminal in each data collection cycle through the sensor, the perception data including the number of gazers, gaze time and face image; Step S2, extracting a face image from the perception data for analysis, wherein the face image includes a left eye two-dimensional coordinate, a right eye two-dimensional coordinate, and a nose two-dimensional coordinate, wherein each face image corresponds to a basic two-dimensional coordinate, and a reference graph is constructed by connecting the left eye two-dimensional coordinate, the right eye two-dimensional coordinate, and the nose two-dimensional coordinate; Step S3, judging whether the face is looking at the information display terminal according to the reference pattern, constructing a horizontal reference line of the face using the connection line of the left eye two-dimensional coordinates and the right eye two-dimensional coordinates, and using the two-dimensional coordinates of the nose as a perpendicular line to the horizontal reference line of the face, and then constructing a vertical reference line of the face, and judging whether the vision of the current face image is a valid gaze angle in combination with the horizontal reference line of the face; Step S4, calculating the number of gazers who meet the effective gaze requirement within the data collection period, and calculating the total effective gaze time within the data collection period according to the gaze time corresponding to the number of gazers, and combining the timestamp corresponding to the data collection period to perform data statistics on the information displayed in the information display terminal, the data statistics including the number of gazers and the total gaze time corresponding to the information; Step S5: evaluating the gaze effect of each display information according to the number of gazers that meet the effective gaze angle and the total gaze time during the data collection period.

2. According to claim 1, a method for intelligent product data analysis based on the Internet of Things is characterized by: The specific steps of step S1 are as follows: Step S1-1, set the data collection period T. When using the data collection period T, mark the corresponding timestamp, specifically, the time of the data collection period T corresponds to an actual timestamp in real time; Step S1-2, obtaining the perception data in the information display area of ​​the information display terminal through the image sensor within the data collection period T, wherein the information display area of ​​the information display terminal is represented as an area that can be contacted by the user's line of sight and can view the content displayed by the information display terminal, and the perception data includes the number of gazers, the gaze time and the face image, and the gaze time is represented as the time from the start of gaze to the end of gaze of a gaze unit gazing at the information display terminal.

3. The method for intelligent product data analysis based on the Internet of Things according to claim 2 is characterized in that: The specific steps of step S2 are as follows: Step S2-1, extracting a face image according to the perception data collected by the image sensor, wherein the face image includes a left eye two-dimensional coordinate, a right eye two-dimensional coordinate and a nose two-dimensional coordinate, and one face image corresponds to one basic two-dimensional coordinate, which is specifically expressed as constructing an independent two-dimensional coordinate for each face image, wherein the abscissa represents the horizontal position in the face image, and the ordinate represents the vertical position in the face image, and the nose two-dimensional coordinate is taken as the origin coordinate, and the left eye two-dimensional coordinate is obtained as (x L ,y L ), the two-dimensional coordinates of the right eye are (x R ,y R ), the two-dimensional coordinates of the nose are (x0, y0); Step S2-2, by connecting the two-dimensional coordinates of the left eye and the two-dimensional coordinates of the right eye, the two-dimensional coordinates of the left eye and the two-dimensional coordinates of the nose, and the two-dimensional coordinates of the right eye and the two-dimensional coordinates of the nose, a geometric structure consisting of three line segments is formed as a reference figure for facial features.

4. The method for intelligent product data analysis based on the Internet of Things according to claim 3 is characterized in that: In step S3, the specific steps of judging whether the face is looking at the information display terminal according to the reference pattern are as follows: Step S3-1, construct the horizontal reference line of the face using the line connecting the two-dimensional coordinates of the left eye and the two-dimensional coordinates of the right eye, use the two-dimensional coordinates of the nose as the perpendicular line to the horizontal reference line of the face, and then construct the vertical reference line of the face, and record the perpendicular point as the vertical reference point of the face, the two-dimensional coordinates are (x N ,y N ); Step S3-2: Set the head turning deviation threshold, input multiple groups of face image data into the neural network model for training, and obtain the basic threshold E for head turning error judgment base , according to the basic threshold E base Calculate and set the head turn deviation threshold. The following formula is used to calculate and set the head turn deviation threshold: ； In the formula, E adj It is represented as the head turning deviation threshold; E base It represents the basic threshold of head turning error judgment obtained through neural network model training; m represents the adjustment coefficient, the sensitivity of confidence to the head turning deviation threshold; C represents the confidence; Step S3-3, judging whether the face image is facing the information display terminal according to the two-dimensional coordinates of the facial vertical reference point, respectively calculating the distance from the facial vertical reference point to the left eye and the distance to the right eye, and calculating according to the Euclidean distance formula, which is as follows: ； Where D represents the Euclidean distance between two points, that is, the calculated straight-line distance; x1, y1 represent the two-dimensional coordinates of the first point, and the two-dimensional coordinates are specifically (x1, y1); x2, y2 represent the two-dimensional coordinates of the second point, and the two-dimensional coordinates are specifically (x2, y2); The left eye 2D coordinate (x L ,y L ) and the facial vertical reference point (x N ,y N ) Substitute into the Euclidean distance formula to get the straight-line distance D from the vertical reference point of the face to the left eye L ; The right eye 2D coordinate (x R ,y R ) and the facial vertical reference point (x N ,y N ) Substitute into the Euclidean distance formula to get the straight-line distance D from the vertical reference point of the face to the left eye R ; The vertical reference point of the face (x N ,y N ) and the two-dimensional coordinates of the nose (0, 0) are substituted into the Euclidean distance formula to obtain the straight-line distance D0 from the vertical reference point of the face to the nose; The left eye 2D coordinate (x L ,y L ) and the right eye 2D coordinate (x R ,y R ) Substitute the Euclidean distance formula to get the straight-line distance D from the left eye to the right eye L-R , and extract the maximum distance D between the left eye and the right eye in the current data collection period T max , and the distance of the nose from the vertical reference point of the face (x N ,y N ) of the maximum distance D max-0 ; Step S3-4, constructing an orthographic two-dimensional coordinate system with the geometric center of the information display terminal as the origin, wherein the abscissa represents the horizontal direction and the ordinate represents the vertical direction, mapping the face image in the orthographic two-dimensional coordinate system, classifying the face image with an abscissa of 0 as an orthographic type, and classifying the face image with an abscissa not equal to 0 as a non-orthographic type; Step S3-4-1: In the emmetropia type, according to the maximum distance D between the left eye and the right eye max And the actual calculated straight line distance D L-R Judge the head turn; When D max =D L-R When the current face image is facing the information display terminal, it is determined that the current face image is facing the information display terminal; When D max >D L-R When the current face image is judged as a non-frontal information display terminal; Step S3-4-2: In the non-emmetropia type, according to the straight-line distance D between the vertical reference point of the face and the left eye L The straight-line distance D from the vertical reference point on the face to the left eye R Judge the head turn; When |D L - D R | ≤ E adj , it is determined that the gaze angle of the face image is a frontal view of the information display terminal; When |D L - D R | > E adj At this time, it is determined that the viewing angle of the face image is a non-frontal viewing information display terminal.

5. The method for intelligent product data analysis based on the Internet of Things according to claim 4 is characterized in that: In step S3, it also includes: Step S3-5, judging whether the face image is looking up or down based on the initial two-dimensional coordinate (0, 0) of the nose and the vertical coordinate of the nose acquired in real time; When D0=D max-0 When the vertical coordinate of the nose rises, it is judged that the initial face image is facing the information display terminal. When the vertical coordinate of the nose rises, it is judged that the face image is looking up. When the vertical coordinate of the nose falls, it is judged that the face image is looking down. When D0<D max-0 When D max-0 Time vertical coordinate; When D max-0 When the vertical coordinate of is a negative value, it is judged that the initial face image is head-up; When D max-0 When the vertical coordinate of is positive, it is judged that the initial face image is looking down; Step S3-6: Setting the head turning threshold Q max , according to the data in the non-orthographic type, the viewing angle after turning the head is calculated to screen out effective gazes. The calculation uses the following formula: ； In the formula, Q avg It is represented by the viewing angle of the face image and the information display terminal; When Q avg ≤Q max When , it is judged that the face image turns its head as effective gaze; When Q avg >Q max When the face image turns its head, it is judged as invalid gaze.

6. The method for intelligent product data analysis based on the Internet of Things according to claim 5, characterized in that: The specific steps of step S4 are as follows: Step S4-1, setting the fixation time threshold S max , in the effective fixation of the face image, filter out the fixation time greater than the fixation time threshold S max The number of people looking at the face, and the corresponding looking time of each face image; Step S4-2: Perform statistical analysis on the content displayed in the information display terminal according to the actual timestamp corresponding to the data collection period T, wherein the number of gazers refers to the number of people who are within the effective gaze range and whose gaze time exceeds the set threshold S max The total fixation time refers to the cumulative total fixation time of face images that meet the effective fixation range and fixation time conditions.

7. The method for intelligent product data analysis based on the Internet of Things according to claim 6, characterized in that: In step S5, the gaze effect of each display information is calculated using the number of annotations in the data collection period T and the total gaze time. The gaze effect calculation uses the following formula: ； Where P i It is represented by the gaze effect score of the i-th displayed information on the information display terminal; w1 is the weight of the number of gazers; w2 is the weight of the gaze time; N i It is represented by the actual number of people who look at the i-th displayed information; t i It is represented as the total fixation time of the i-th displayed information; N max It is expressed as the maximum number of historical gazes; t max It is expressed as the maximum value of the sum of historical fixation time; Display resources are allocated to the display information of the information display terminal according to the gaze effect calculation result.

8. A product data intelligent analysis system based on the Internet of Things, which is applied to a product data intelligent analysis method based on the Internet of Things according to any one of claims 1 to 7, characterized in that: The intelligent analysis system includes a data acquisition module, a face image analysis module, a gaze angle judgment module, a gaze statistics module and a gaze effect evaluation resource allocation module; The data acquisition module is used to regularly collect user perception data, including the number of gazers, gaze time and face images; the face image analysis module is used to extract the user's facial image data, analyze the coordinates of the eyes and nose, and construct a reference graph; the gaze angle judgment module is used to judge whether the gaze is at the information display terminal and whether it meets the effective gaze angle according to the face image analysis result; the gaze statistics module is used to count the number of gazers and the total gaze time within the effective gaze range; the gaze effect evaluation resource allocation module is used to evaluate the gaze effect of the display content according to the statistical data and reasonably allocate display resources; The output end of the data acquisition module is electrically connected to the input end of the face image analysis module; the output end of the face image analysis module is electrically connected to the input end of the gaze angle judgment module; the output end of the gaze angle judgment module is electrically connected to the input end of the gaze statistics module; the output end of the gaze statistics module is electrically connected to the input end of the gaze effect evaluation resource allocation module.

9. The product data intelligent analysis system based on the Internet of Things according to claim 8 is characterized by: The data acquisition module includes a data acquisition cycle setting unit and a perception data acquisition unit; the data acquisition cycle setting unit is used to set and adjust the data acquisition cycle T; the perception data acquisition unit is used to acquire the perception data in the information display area of ​​the information display terminal through the image sensor within a specified period; The face image analysis module includes a face image extraction unit and a reference graph construction unit; the face image extraction unit is used to extract the face image of each user from the perception data, and obtain the coordinates of the left eye, the right eye and the nose in the two-dimensional coordinate system; the reference graph construction unit is used to construct a reference image; The gaze angle judgment module includes a baseline construction unit and a gaze angle judgment unit; the baseline construction unit is used to construct a horizontal baseline of the face and a vertical baseline of the face; the gaze angle judgment unit is used to judge whether the face is looking at the information display terminal.

10. The product data intelligent analysis system based on the Internet of Things according to claim 8, characterized in that: The gaze statistics module includes a gaze number statistics unit and a gaze time statistics unit; the effective gaze range statistics unit is used to count the number of annotations; the gaze time statistics unit is used to count the gaze time; The gaze effect evaluation resource allocation module includes a gaze effect evaluation unit and a display resource allocation unit; the gaze effect evaluation unit is used to calculate the gaze effect of each display information according to the number of gazers and the gaze time; the display resource allocation unit is used to reasonably allocate the display resources of the information display terminal according to the gaze effect score.

Images