Exhibition exhibit attention recognition system based on computer vision

Abstract

This invention discloses a computer vision-based exhibition exhibit attention recognition system, specifically in the field of behavior analysis, comprising: a front-end perception module, an edge computing module, and an attention evaluation module. The front-end perception module completes camera equipment deployment, joint camera calibration, and exhibit parameter pre-definition, outputting a synchronized video stream and spatial calibration data after temporal synchronization and transmission buffering. The edge computing module, based on the video stream, performs human detection and tracking, skeleton key points, and pose depth calculation, identifies audience interaction actions and mobile phone interference, and generates standardized human feature vectors. The attention evaluation module reuses calibration data and human features to complete audience-exhibit spatial association and optimal matching. Through session management, multi-factor score calculation, and occlusion compensation, it smoothly outputs exhibit attention heatmap values ​​in a temporal sequence and retains behavioral event records. This invention achieves automated, refined, and quantitative recognition of exhibition exhibit attention, adapting to real-time analysis application scenarios with no blind spots in exhibition halls.

Description

Technical Field

[0001] This invention relates to the field of behavioral analysis technology, and more specifically, to a computer vision-based system for recognizing attention to exhibits. Background Technology

[0002] With the large-scale and routine operation of various cultural and museum exhibition halls, commercial exhibition halls, and science exhibitions, venue operators generally need to monitor the actual visitor flow and audience preferences for different exhibits in real time. This is crucial for optimizing exhibit layout, allocating on-site maintenance personnel, and adjusting the pace of exhibition operations. Currently, the mainstream data collection methods in the industry rely heavily on manual on-site inspections and registrations, traditional visitor flow statistics, and fixed-point infrared sensor counting. These methods can only roughly estimate the total number of people passing through an area and cannot accurately distinguish whether visitors are truly interested in the corresponding exhibits, nor can they identify the visitors' focus, lingering behavior, and willingness to interact during the exhibition.

[0003] Meanwhile, conventional statistical methods are easily affected by complex on-site environments such as dense crowds, cross-movement of people, and irrelevant passersby. Data collection is not timely, labor costs are high, statistical dimensions are limited, and the level of detail is insufficient. These methods cannot meet the needs of modern smart exhibition halls for unmanned, all-day, and high-precision analysis of exhibit attention. The industry lacks supporting technologies for intelligent attention measurement and identification that can adapt to complex exhibition scenarios and operate stably around the clock.

[0004] However, it still has some drawbacks in practical use, such as:

[0005] 1. The data collection is extremely low in precision. Existing conventional visitor flow statistics methods can only complete a rough summary of the number of people passing through the area. They cannot accurately distinguish the different behaviors of visitors, such as stopping to watch exhibits, passing by quickly, or staying to chat. They cannot truly reflect the actual attention paid to a single exhibit. The statistical data is coarse-grained and cannot provide effective detailed data support for optimizing the exhibition layout. The ability to adapt to the refined operation of smart exhibition halls is seriously insufficient.

[0006] 2. High manual operation and maintenance costs: Traditional exhibition hall attention surveys rely heavily on staff on-site at fixed points, segmented patrols, and manual record-keeping. This requires a large workforce around the clock, resulting in high long-term economic costs. Furthermore, manual recording is susceptible to subjective judgment, leading to frequent omissions, errors, and delays in registration. The low tolerance for error makes it difficult to guarantee the standardization and reliability of the overall data collection process.

[0007] 3. The equipment has weak anti-interference ability in complex on-site environments. Traditional infrared sensing and access control card-swiping equipment have simple working logic. When faced with complex real-world conditions such as dense crowds in exhibition halls, people crossing back and forth, and foreign objects blocking sensor points, it is very easy to encounter abnormal problems such as counting errors, signal loss, and duplicate statistics. It has poor environmental adaptability, insufficient ability to operate continuously and stably around the clock, and high frequency of invalid abnormal data.

[0008] 4. The data is limited in scope and lacks real-time performance. Existing technologies can only output basic visitor flow data and cannot simultaneously collect multi-dimensional related data such as visitor dwell time, orientation, and interaction behavior, resulting in a one-sided data analysis. Furthermore, the data is mostly aggregated and archived in batches after the fact, making it impossible to synchronize real-time dynamic monitoring on-site within seconds. It also hinders the rapid coordination and allocation of on-site operation and maintenance resources, severely lacking in intelligent and real-time management efficiency. Summary of the Invention

[0009] In order to overcome the above-mentioned defects of the prior art, the present invention provides an exhibition exhibit attention recognition system based on computer vision, which solves the problems mentioned in the background art through the following solutions.

[0010] To achieve the above objectives, the present invention provides the following technical solution: a computer vision-based exhibition exhibit attention recognition system, comprising:

[0011] Front-end perception module: Standardized deployment of multi-view image acquisition equipment, completion of joint calibration of camera's full-domain internal and external parameters and construction of a unified spatial coordinate system on site, pre-set three-dimensional spatial geometric information of exhibits and image association areas, completion of hardware timing synchronization of multi-channel visual acquisition equipment, acquisition of panoramic visual data on site, and synchronous output of unified timing video stream, calibration spatial parameters and pre-set geometric data of exhibits;

[0012] Edge computing module: It interfaces with the time-synchronized video stream and basic spatial parameters output by the front-end perception module, continuously detects human targets in the field and tracks them across frames, extracts key points of the human skeleton and calculates the three-dimensional spatial position of the hands in combination with unified spatial parameters, calculates the head posture and completes the global coordinate system calibration, solves the spatial depth information and the degree of mutual occlusion of the human body in real time, judges the on-site photo-taking action, pointing action and mobile phone occlusion conditions, and outputs a complete set of human perception feature data in a structured manner.

[0013] Attention Assessment Module: This module retrieves pre-set geometric data of exhibit spaces from the front-end perception module, integrates complete human perception feature data output by the edge computing module, performs layer-by-layer verification and screening of the association between the human body and exhibit spaces, combines head orientation to complete association matching and judgment, executes optimal allocation and binding for each person and exhibit, establishes a time-sharing exhibition session sequence management mechanism, integrates posture stability, effective dwell time, line-of-sight matching, and multi-dimensional interactive behavior to comprehensively calculate individual attention, overlays on on-site occlusion conditions to complete score compensation and correction, periodically aggregates and calculates and outputs standardized attention heat values ​​for each exhibit, and simultaneously generates a complete archive record of exhibition attention events.

[0014] The technical effects and advantages of this invention are as follows:

[0015] 1. Full-domain refined and seamless intelligent analysis: Relying on a visual full-domain perception architecture, it can collect and analyze the behavior of all visitors in the venue without them having to wear any auxiliary identification equipment. It can accurately distinguish between effective viewing and ineffective passing behavior, quantify the real attention of each exhibit, and provide fine data granularity. It can fully cover and adapt to the needs of simultaneous analysis of multiple points in the entire exhibition hall, in line with the unmanned operation standards of smart exhibition halls.

[0016] 2. All-day cost reduction and efficiency improvement with stable operation, completely replacing the traditional manual inspection and registration process, significantly reducing on-site manpower costs and paper ledger management costs. The equipment is equipped with edge lightweight computing mode, which can work continuously 24 hours a day without interruption, without fatigue operation or human error, and greatly improve the overall work efficiency and standardization of attention statistics.

[0017] 3. Strong anti-interference and adaptability in high-density crowd flow. Equipped with multi-level spatial verification and adaptive occlusion compensation mechanism, it can stably complete target recognition and feature analysis in complex real-world conditions such as peak crowd congestion, overlapping and occlusion of people, and random shuttle interference in the exhibition hall. It effectively avoids data distortion caused by environmental interference and is suitable for various high and low peak all-time exhibition hall operation scenarios.

[0018] 4. Multi-dimensional synchronous output and real-time linkage empowerment: It synchronously integrates multiple dimensions of analysis, such as dwell time, posture status, interaction frequency, and heat distribution, providing complete and comprehensive data information. It also supports the real-time output of exhibit popularity results in seconds and can directly link with the back-end exhibition hall management platform to quickly assist operation and maintenance personnel in allocating resources, optimizing exhibitions, and guiding visitor flow, thereby comprehensively improving the overall intelligent operation and management level of the exhibition hall. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall structure of the present invention.

[0020] Figure 2 This is a schematic diagram of the front-end perception module of the present invention.

[0021] Figure 3 This is a schematic diagram of the edge computing module of the present invention.

[0022] Figure 4 This is a schematic diagram of the attention assessment module of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] refer to Figure 1 - Figure 4 The computer vision-based exhibition exhibit attention recognition system shown includes:

[0025] Front-end perception module: This module serves as the system's underlying data entry point and spatial calibration foundation. Through standardized deployment of multi-view cameras, joint calibration of internal and external parameters, and pre-definition of exhibit 3D geometry and image ROI, combined with PTP high-precision time synchronization and gigabit edge transmission, it completes the acquisition of multi-channel synchronous video streams and the construction of a global spatial coordinate system at the exhibition site. This provides a unified calibration benchmark and original visual data source for subsequent human feature extraction, coordinate back projection, and target-exhibit spatial association, achieving integrated front-end support for perception hardware, spatiotemporal synchronization, and spatial modeling.

[0026] This module first standardizes the deployment of on-site visual acquisition hardware, clarifying the camera installation locations, angles, and hardware parameter specifications to ensure blind-spot-free coverage of the entire exhibition area. Then, it establishes a unified world coordinate system through joint calibration of camera intrinsic and extrinsic parameters, simultaneously predefining the 3D space, normal direction, and image ROI region for each exhibit. Finally, relying on hardware time synchronization and wired transmission links, it achieves time alignment of multiple video streams, stable backhaul, and short-term edge caching, providing regular and reliable raw input data for backend algorithm processing.

[0027] It should be further explained that the camera deployment specifically includes:

[0028] The system is equipped with at least two RGB cameras for each exhibit area to form complementary perspectives, respectively undertaking the tasks of frontal eye-level and top-down view acquisition:

[0029] Front-facing horizontal camera: Installation height controlled between 1.6 and 2.0 meters, with a downward angle set to [0°, 15°];

[0030] Ceiling-mounted overhead camera: The installation height should be controlled between 3.0 and 5.0 meters, and the shooting angle should be vertically downward or diagonally downward at 30°.

[0031] The hardware parameters for each camera are uniformly set as follows: horizontal field of view ≥ 120°, representing the range of field of view that the camera can cover laterally; resolution 1920×1080, representing the pixel sampling scale of a single frame image; fixed frame rate of 30fps, representing the number of image frames acquired per second to ensure the continuity of human movement sequence. All cameras are connected to the PTP hardware time synchronization mechanism, with a timestamp accuracy ≤ 1ms. The timestamp accuracy represents the upper limit of the timing deviation between frames acquired by multiple cameras, ensuring strict timing alignment of multiple video frames at the hardware level.

[0032] Further explanation is needed regarding the calibration parameters, which specifically include:

[0033] After the hardware deployment is completed, single-camera intrinsic parameter calibration and multi-camera joint extrinsic parameter calibration are carried out sequentially: first, the intrinsic parameter matrix of each camera is solved. Image distortion correction is achieved by combining lens distortion coefficients; then, through joint extrinsic parameter calibration, the rotation matrix of each camera in the global world coordinate system is calculated. Translation vector .

[0034] The system uniformly agrees on coordinate transformation rules: points in the world coordinate system Transformation to camera coordinate system follows .

[0035] For each exhibit Define multi-dimensional geometric information in advance, including:

[0036] Center point of exhibits in three-dimensional space ;

[0037] Unit vector of exhibit normal direction pointing to the main audience area ;

[0038] Image plane ROI polygon vertex set obtained by coordinate projection mapping , , These are the horizontal and vertical coordinates of the image pixel plane, respectively;

[0039] The three-dimensional spatial enclosure area of ​​the exhibit used for subsequent close-range touch detection. .

[0040] It should be further explained that data acquisition and transmission specifically include:

[0041] After calibration and parameter pre-configuration, the system enters normal data acquisition mode. Each camera continuously outputs RGB image frames with time-stamped tags. Each frame is bound to an acquisition timestamp t, which represents the physical acquisition time of a single frame image, along with the camera number c, facilitating backend traceability and segmentation processing. The acquired raw video data is stably transmitted to the edge computing server via gigabit Ethernet. A local caching strategy is also configured, where raw images are only temporarily retained at the edge for no more than 10 seconds, saving storage resources while meeting the algorithm's real-time retrieval requirements.

[0042] Furthermore, this module outputs synchronized multi-channel video streams. , as well as all predefined calibration parameters and exhibit geometry information.

[0043] Edge computing module: Receives synchronous multi-channel video streams from module one, performs human target detection and continuous cross-frame tracking frame by frame, locks all human targets within the frame and assigns a unique identifier; extracts key points of the human skeleton, completes effective key point selection and hand 3D coordinate back-projection calculation; performs head pose estimation and transforms to the world coordinate system, unifying the global orientation representation; achieves relative depth to physical true depth mapping through monocular depth estimation, quantifying the occlusion ratio between human bodies; the fifth step independently analyzes the binary states of taking photos, pointing actions, and mobile phone occlusion, completing behavioral feature extraction without relying on prior association with exhibits; finally, integrates multi-dimensional information to generate standardized human feature vectors, aligns them temporally, and sends them to module three, providing complete feature input for attention assessment.

[0044] It needs to be further described that target detection and tracking specifically includes:

[0045] First, precise global human target detection and continuous cross-frame tracking are performed on each frame of the real-time input raw image. The backbone detection algorithm uses a lightweight, high-precision YOLOv8 detector, and the bounding boxes of all valid human targets in the image are regressed frame by frame. , , These represent the horizontal and vertical coordinates of the top-left corner of the bounding box of the human target pixels, respectively. , These represent the horizontal and vertical coordinates of the bottom right corner of the human target pixel bounding box, respectively. Real-time detection of reliable probability values ​​for individual human targets.

[0046] Synchronous access after detection This lightweight multi-target tracker integrates Kalman filtering for real-time motion state prediction with high-precision cascaded bidirectional matching. It assigns a unique, globally specific ID to each valid human target in the frame, achieving seamless cross-frame target association and binding. The system includes a standardized target lifecycle management mechanism: newly appearing human targets are automatically assigned new IDs; if a disappeared human target fails to complete a valid match association for 30 consecutive frames, its corresponding ID is automatically cancelled, avoiding the consumption of computing resources by invalid and redundant identifiers. Output: A list of all valid human targets in each frame. And a unique tracking ID.

[0047] It should be further explained that the key point extraction of the human skeleton and the calculation of the three-dimensional coordinates of the hand specifically include:

[0048] Based on precise human target tracking, it is equipped with high-precision and lightweight components. The skeleton extraction model calculates the coordinates and reliable parameters of 17 standard COCO format joint keypoints of the human body in a single frame, and outputs the raw pixel information of a single set of standard keypoints uniformly. , , To correspond to the precise horizontal and vertical coordinates of key human joint points in the image pixel plane, The real-time detection confidence reliability of the corresponding key joint points.

[0049] Based on subsequent algorithm analysis of essential needs, key points of the core torso are selectively screened and synthesized: the average value of symmetrical key points on the left and right shoulders is used to fit and solve for the shoulder center. The mean value of the key points symmetrical to the left and right hips is used to fit and solve for the hip center. Simultaneously and directly extract specific and effective key points on both sides of the wrist. , .

[0050] Strictly set a pre-screening constraint threshold for skeleton validity: when the real-time confidence of the key point corresponding to any shoulder is lower than 0.5, the full skeleton acquisition information of the human target is directly determined to be invalid, and all subsequent three-dimensional feature calculation processes are automatically skipped. Invalid targets do not occupy edge computing power.

[0051] For verified and valid skeletal human targets, a high-precision three-dimensional spatial coordinate back-projection calculation of the hand was carried out: relying on the pixel coordinates of the original image of the wrist's corresponding key points. , , For the effective key points on the wrist, match the horizontal and vertical coordinates of the corresponding image pixel plane with the actual physical depth values ​​of the corresponding points. , To correspond to the actual physical spatial depth distance of each pixel location, and in conjunction with the complete set of camera intrinsic and extrinsic geometric models calibrated at the front end, the two-dimensional plane pixel coordinates are accurately reverse-mapped to the global unified world coordinate system. The core conversion formula is as follows:

[0052]

[0053] Based on the inherent orthogonality of rotation matrices, direct equivalent conversion satisfies The global depth values ​​are uniformly sampled directly from the corresponding pixel points in the real-time depth map. If the real-time confidence level of the key point corresponding to the wrist is lower than 0.5, the three-dimensional spatial coordinates of the hand are directly marked as invalid and usable.

[0054] Output: 3D left wrist space coordinates for each valid tracked target and three-dimensional right wrist spatial coordinates Simultaneously, a valid verification marker for the location is attached.

[0055] It should be further explained that head pose estimation and coordinate system transformation specifically include:

[0056] To accurately determine the real-time focus direction of the audience's gaze and adapt to the subsequent exhibit association matching needs, a high-precision 6D-RepNet head pose regression model is used to directly calculate and output the head pose rotation matrix in the camera's dedicated coordinate system from a single frame of RGB real-scene image. .

[0057] Based on the inherent spatial transformation properties of the head pose rotation matrix, the real-time spatial orientation vector of the viewer's head in the camera-specific coordinate system is derived and solved. .

[0058] To adapt to the unified spatial assessment standard across the entire exhibition hall and eliminate interference from multi-camera perspective deviations, the original head orientation vector in the camera coordinate system is uniformly transformed and aligned to the global world coordinate system. This ensures that pose data across all scenes and perspectives is available from the same source. The coordinate transformation formula is as follows:

[0059] ;

[0060] Output: Global world coordinate system head orientation vector for each human target after alignment in each frame. .

[0061] It should be further explained that depth estimation and occlusion detection specifically include:

[0062] It accurately acquires two core parameters: the real-world spatial distance of human figures in the entire scene and the degree of mutual occlusion among pedestrians, providing a basis for subsequent score compensation, and adopts industrial-grade... A monocular high-precision depth estimation network performs dense depth analysis pixel by pixel on the entire image frame and outputs a normalized relative depth heatmap across the entire domain. , For any image Assign a dimensionless normalized relative depth value to the pixel location.

[0063] To eliminate the dimensionless bias of relative depth and align with the physical judgment standards of the actual scene, linear regression calibration was carried out in advance using physical reference objects of known standard dimensions on site to accurately solve for the global fixed depth scale factor s and the depth reference translation. A linear mapping closed-loop model from relative depth to standard physical depth in the real scene is constructed, and the conversion formula is as follows:

[0064]

[0065] In the formula Strictly select the pixel position corresponding to the center of the human target bounding box. , The horizontal and vertical coordinates of pixels are projected onto the image plane from the geometric center of the human target. The depth sampling of key points on the wrist uses the same conversion standard throughout the process to ensure that the dimensions of depth parameters are consistent and the caliber is uniform across the entire scene.

[0066] After deep calculation, the degree of interference from overlapping occlusion of multiple pedestrians on site is accurately quantified. By calculating the percentage of overlapping pixel area of ​​multiple human target bounding boxes in real time, and combining the depth and near-far hierarchical relationship of the real scene of the targets in front and behind, the real-time comprehensive occlusion ratio of a single human target is normalized and solved. The core calculation formula is as follows:

[0067]

[0068] Each adjacent interfering human body within the image, excluding the currently identified target, is assigned a unique identifier. This represents the total pixel area of ​​the real-time overlapping region between the two sets of human body bounding boxes. To determine the total area of ​​the complete bounding box of a human target, an indicator function is used. Used for logical determination of whether surrounding interfering targets are in the real-world space in front of the current target.

[0069] Output: The true depth of the real scene for each human figure in each frame. With normalized occlusion ratio , The quantification characterizes the severity of the human target being occluded by adjacent pedestrians in front; the closer the value is to 1, the stronger the occlusion interference.

[0070] It should be further clarified that the specific actions and phone occlusion states include:

[0071] After all basic spatial visual features are calculated, general human interaction actions and handheld device interference states are independently and offline analyzed. This step does not require association with any exhibit space information, achieving lightweight extraction of behavioral features in advance and reducing the computational burden on the backend. Intelligent judgment of photo-taking actions: Real-time verification of the hand key point lifting height matching the eye interval threshold, simultaneous recognition of the outline features of rectangular electronic devices such as mobile phones in the image, accurate capture of the rising edge of the action, and assignment of a mark when a valid action is triggered. If no action is taken, set the value to 0. A binary special flag to characterize whether a single human target has engaged in photo-taking interaction at any given moment.

[0072] Intelligent motion analysis: Simultaneously verifying triple constraints—a fully extended single arm, a wrist position exceeding the shoulder horizontal reference plane, and a continuous motion lasting at least 3 frames—accurately captures the start moment of the motion and assigns output values ​​upon effective triggering. If no compliance action is taken, the value will be set to 0. A dedicated binary flag to characterize whether a single human target instantly interacts with exhibits.

[0073] Mobile phone occlusion steady-state analysis: Continuously inspect local image areas around the hand, identify rectangular occlusion objects such as mobile phones in real time, add a 5-frame continuous steady-state verification threshold, and mark the current frame after the threshold is met. , The binary discrimination flag is used to represent the momentary operation of the audience's mobile phone and the interference with effective focused viewing. It is specifically used for the accurate selection of the effective dwell time. It only makes steady-state judgments and does not capture the rising edge of the action.

[0074] Specific technical notes: Near-field touch interaction events require the judgment of the boundary of the exhibit's exclusive three-dimensional spatial area, which depends on the precise binding results of the three objectives of the module - exhibit. Therefore, this module does not perform touch event parsing.

[0075] Output: Temporal alignment state parameters for each human target in each frame. , , .

[0076] Furthermore, the final output of this module is: for each frame acquisition time t and each valid human tracking target ID, all parsed structured features are fully integrated and encapsulated to generate a standardized high-dimensional specific feature vector.

[0077]

[0078] Attention Assessment Module: This module loads predefined geometric parameters of exhibits from Module 1 and human feature vectors from Module 2. The first step reuses coordinate transformation rules to solve for the global position of the human target. The second step uses image ROI constraints and physical depth filtering to achieve a preliminary spatial association between the audience and exhibits. The third step constructs association probabilities based on the angle between head orientation and exhibit normals, using the Hungarian algorithm to achieve optimal spatiotemporal matching between the audience and exhibits. The fourth step establishes audience-exhibit session lifecycle management, updating effective dwell time and various interaction behavior counts frame by frame. The fifth step integrates multiple factors such as gaze, posture stability, dwell time, and interaction behavior to calculate the instantaneous attention in a single frame, and overlays the crowd occlusion ratio to complete score compensation correction. The sixth step aggregates the individual audience score, updates the overall heat value of the exhibits through a fixed-period sliding average, generates a structured attention event log after the session ends, and finally outputs exhibit heat data and behavioral event records, completing a closed loop of full-process attention measurement.

[0079] It should be further explained that the predefined data and coordinate transformation specifically include:

[0080] The system first loads the pre-calibrated static geometric parameters of the entire exhibition hall in batches, and then retrieves the three-dimensional center point corresponding to each exhibit j. , Exhibit front normal orientation vector Image projection ROI region outline 3D entity bounded area determination region The synchronous reuse module utilizes a mature pixel back-projection calculation paradigm to reverse-calculate the center point of the human target detection box from the pixel position of the two-dimensional image to the global three-dimensional space, outputting the real-time spatial position of the audience. , It represents the actual physical spatial coordinates of the audience's center of mass in the overall three-dimensional coordinate system of the exhibition, with a unified coordinate caliber across the entire area to ensure unbiased data exchange between modules.

[0081] It should be further explained that the determination of the correlation between the target and the exhibition space specifically includes:

[0082] Based on a unified world coordinate system, a two-layer physical space joint constraint verification is performed frame by frame on all audience target IDs and all exhibits j to quickly eliminate invalid associations in distant or uncovered areas, completing the coarse matching pre-screening:

[0083] Image-level ROI compliance verification: Determining the projection pixel coordinates of the viewer's target center Does it fall within the exhibit's designated effective imaging area? Within the region, spatial relationships are initially determined to be valid, and values ​​are assigned. ;

[0084] Physical depth verification: For preliminary valid pairings, calculate the difference between the audience's actual spatial depth and the reference depth of the exhibit's center. The exhibit's center depth is obtained by converting the exhibit's 3D center point to the camera's Z-axis. If the spatial depth difference is greater than 2 meters, it is determined to be a false detection interference from distant views, and the value is forcibly zeroed out. This effectively avoids the problem of unrelated exhibits being mistakenly associated across regions.

[0085] It should be further explained that the head orientation angle error specifically includes:

[0086] Only exhibit pairings that pass both spatial verifications are retained. Further verification of attention matching is performed from a visual gaze perspective. A spatial vector dot product algorithm is used to accurately calculate the spatial angle deviation between the viewer's actual gaze direction and the exhibit's frontal orientation.

[0087]

[0088] The physical meaning is the spatial angle deviation between the viewer's line of sight and the normal of the exhibit. The smaller the angle, the more directly the viewer's line of sight is directed at the exhibit and the stronger their intention to pay attention. The algorithm automatically converts the radian into an angle for easy engineering calculation. At the same time, a hard 90° cutoff threshold is set. If the angle exceeds the limit, it will be capped at 90° to prevent reverse lines of sight or irrelevant lines of sight from the side from causing abnormal and distorted scoring.

[0089] It should be further explained that the initial association probability specifically includes:

[0090] Using the viewing angle deviation as the core evaluation criterion, a Gaussian probability distribution function is introduced to quantify the confidence level of the fit between the audience and the exhibits, and to construct a refined association weight:

[0091] ,

[0092] Unqualified spatial pairings are directly forced to have their probability reset to zero, while qualified pairings are adaptively output with a continuous probability value in the range of 0 to 1 based on the angle deviation. The higher the value, the more likely the visitor is to pay attention to the corresponding exhibit, providing a precise weighting basis for subsequent global optimal matching.

[0093] It should be further explained that the exclusive allocation of spacetime specifically includes:

[0094] To address the issues of cross-interference between multiple visitors and exhibits, and duplicate binding of one person to multiple exhibits, a target-exhibit bipartite graph matching topology is constructed. Visitor IDs are designated as left-hand nodes, exhibit numbers as right-hand nodes, and edge weights between nodes correspond to association probabilities. The Hungarian algorithm is introduced to solve for the global maximum weighted matching result, with strong uniqueness constraints to strictly ensure that the same visitor can only be assigned to one exhibit at a time, preventing multiple exhibits from competing for the same target. The final output is a binary matching label. , This is a frame-specific matching flag. A value of 1 indicates that the viewer in the current frame has been uniquely bound to the exhibit, while a value of 0 indicates that there is no binding relationship. A minimum probability threshold of 0.3 is set simultaneously. Viewers whose probability does not meet the minimum across all dimensions will not be assigned exhibits and will be judged as wandering passersby.

[0095] It should be further explained that the target and exhibit conversation management and individual attention score calculation specifically include:

[0096] For continuous viewing scenarios, an innovative and comprehensive session lifecycle management mechanism has been added. A continuous time segment where the same viewer is continuously bound to the same exhibit is defined as an independent viewing session. A new session begins immediately when the matching flag changes from 0 to 1; the old session ends immediately when the matching is interrupted or the exhibit is switched, and all accumulated session variables are automatically cleared. This achieves physical isolation and prevents crosstalk between data from different viewing periods. The system dynamically attaches unique status variables to each active session, including the effective dwell time. , The physical meaning refers to the cumulative effective time that visitors actually spend attentively viewing the exhibition after eliminating mobile phone interference; the cumulative count of the three types of interactive behaviors. , , These represent the actual cumulative number of times a visitor takes a photo, points at, or touches an exhibit at close range within the session period; a session frame counter and a previous frame touch state memory bit are also configured for behavior edge determination.

[0097] In each frame where the session is ongoing and the match is valid, four sequential iterative updates and intelligent scoring calculations are performed in an orderly manner:

[0098] Effective dwell time update: Validity filtering is performed using the second-hand phone occlusion binary flag from the module. If... This indicates that the audience is not addicted to their phones and is in an effective viewing state, according to the single-frame time interval. , The system uses a fixed single-frame sampling time interval, which is directly determined by the reciprocal of the acquisition frame rate, and accumulates the effective dwell time frame by frame. Accumulation is paused when there is interference from mobile phone operation to ensure that the duration data truly reflects the actual concentration level of the visitors.

[0099] Photo and pointing count updates: The action event flags output by the real-time linkage module 2 will accumulate the photo count when the rising edge of the photo is detected, and will accumulate the pointing count when the pointing action is detected. The frequency of active interaction behavior will be recorded sequentially throughout the process.

[0100] Touch event detection and counting: It relies entirely on three-dimensional spatial coordinates for autonomous determination, without depending on the auxiliary input of other modules. It retrieves the effective three-dimensional coordinates of both wrists output by module two and calculates the distance from the wrist position to the three-dimensional physical area of ​​the exhibit in real time. Shortest Euclidean distance:

[0101]

[0102] Physically, it refers to the shortest physical distance between the viewer's hand and the physical area of ​​the exhibit, used to accurately identify close-range touch behavior; a distance of less than 0.1 meters is considered a valid touch, and the touch status of the current frame is marked. , It serves as a real-time binary status indicator for whether the hand is in contact with the exhibit in the current frame; by comparing the status of previous and next frames to capture the rising edge, the count is incremented only on the first touch to avoid repeated counting of continuous touches and ensure accurate interaction counting.

[0103] Instantaneous attention score calculation: Integrating the core factors of four-dimensional exhibition viewing, and adopting a primary and secondary weighted fusion model, a refined attention score for each frame and target is generated in real time.

[0104]

[0105] The physical meaning is a quantitative score representing the real-time focus of a single viewer on a specific exhibit at a single frame; the weights are fixed. =0.7、 =0.3, focusing on static viewing posture while taking into account active interactive behavior, in line with the actual viewing and cognitive patterns.

[0106] Explanation of the physical meaning of each dimension factor:

[0107] The line-of-sight matching factor has a higher score for the smaller the angle, which directly reflects the degree to which the viewer's line of sight is aligned with the exhibit.

[0108] Posture stability factor: Characterizes the stability of the audience's body posture while viewing the exhibition. It is calculated based on the time series standard deviation of the shoulder horizontal angle within 1 second. The smaller the fluctuation, the more stable the posture, and the higher the score, which is close to the physical characteristics of a real focused viewer.

[0109] The duration contribution factor describes the positive effect of dwell time on attention. It gradually saturates as the dwell time increases to avoid abnormal overflow of scores for long dwell times.

[0110] Active interaction factor: differentiates and quantifies the intensity of various interactive behaviors, with the upper limit normalized to 1, objectively reflecting the audience's willingness to actively explore.

[0111] It should be further explained that crowd occlusion compensation specifically includes:

[0112] To address the real-world challenge of feature extraction distortion caused by mutual occlusion in high-density crowds at the exhibition hall, an adaptive occlusion attenuation compensation mechanism has been added; this mechanism adjusts based on the real-time occlusion ratio of visitors. When the instantaneous score is greater than 0.5 (high occlusion threshold), the instantaneous score is corrected by linear decay according to the degree of occlusion. ;in The physical meaning is the effective instantaneous attention score after correction for occlusion interference; only single-frame adaptive compensation is performed, without additional complex temporal attenuation, balancing the real-time performance of the algorithm and the accuracy of quantization.

[0113] It should be further explained that the update and moving average of exhibit popularity heat values ​​specifically include:

[0114] Aggregate effective audience scores from all scenarios and accumulate them frame-by-frame to generate real-time increase in exhibit attention: ;in The instantaneous cumulative increment of the attention contributed by all valid viewers to the exhibit in the current frame is calculated; to eliminate interference from instantaneous fluctuations in crowd flow and improve output stability, a moving average smoothing filter is performed with a fixed period of 5 seconds and 150 frames.

[0115] , The steady-state average heat value of the exhibit over the complete statistical period directly represents the global level of attention received by the exhibit; T is the end frame number corresponding to the sliding statistical window. The heat value has no fixed upper limit and can be adaptively adapted to different traffic scenarios during peak and off-peak periods.

[0116] After a single exhibition session concludes naturally, the system automatically generates a standardized, structured log of attention events with a single click:

[0117]

[0118] The total valid viewing time for a single complete session is calculated as follows: The average line-of-sight angle throughout the entire conversation. It provides an average audience focus score during the session, fully retains the source data of the entire behavioral chain, and supports backend exhibition big data review and analysis.

[0119] Furthermore, this module outputs: heat values ​​for each exhibit updated every 5 seconds. Full-scale structured records of audience attention events can be directly connected to external operation and maintenance analysis platforms.

[0120] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other.

[0121] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A computer vision-based exhibition exhibit attention recognition system, characterized in that, include: Front-end perception module: Standardized deployment of multi-view image acquisition equipment, completion of joint calibration of camera's full-domain internal and external parameters and construction of a unified spatial coordinate system on site, pre-set three-dimensional spatial geometric information of exhibits and image association areas, completion of hardware timing synchronization of multi-channel visual acquisition equipment, acquisition of panoramic visual data on site, and synchronous output of unified timing video stream, calibration spatial parameters and pre-set geometric data of exhibits; Edge computing module: It interfaces with the time-synchronized video stream and basic spatial parameters output by the front-end perception module, continuously detects human targets in the field and tracks them across frames, extracts key points of the human skeleton and calculates the three-dimensional spatial position of the hands in combination with unified spatial parameters, calculates the head posture and completes the global coordinate system calibration, solves the spatial depth information and the degree of mutual occlusion of the human body in real time, judges the on-site photo-taking action, pointing action and mobile phone occlusion conditions, and outputs a complete set of human perception feature data in a structured manner. Attention Assessment Module: This module retrieves pre-set geometric data of exhibit spaces from the front-end perception module, integrates complete human perception feature data output by the edge computing module, performs layer-by-layer verification and screening of the association between the human body and exhibit spaces, combines head orientation to complete association matching and judgment, executes optimal allocation and binding for each person and exhibit, establishes a time-sharing exhibition session sequence management mechanism, integrates posture stability, effective dwell time, line-of-sight matching, and multi-dimensional interactive behavior to comprehensively calculate individual attention, overlays on on-site occlusion conditions to complete score compensation and correction, periodically aggregates and calculates and outputs standardized attention heat values ​​for each exhibit, and simultaneously generates a complete archive record of exhibition attention events.

2. The computer vision-based exhibition exhibit attention recognition system according to claim 1, characterized in that: The output of the unified temporal video stream, calibration spatial parameters, and preset geometric data of the exhibits includes: Complete the standardized layout and angle adaptation installation of multi-view camera points, carry out camera intrinsic parameter distortion correction and extrinsic parameter joint spatial calibration throughout the entire area, and establish a unified global three-dimensional spatial coordinate system for the exhibition hall; pre-enter the three-dimensional center position, spatial normal direction, image projection area and the spatial range surrounded by the entity for each exhibit, and align the timing of all acquisition devices based on the hardware synchronization mechanism; collect the on-site visual images of the entire exhibition hall in real time, cache the original image data for a short time, and continuously push standardized video data streams, global calibration spatial coordinate parameters and solidified pre-set geometric file data of all exhibits outward for timing alignment.

3. The computer vision-based exhibition exhibit attention recognition system according to claim 1, characterized in that: The continuous detection and tracking of human targets within the field across frames includes: It reads synchronized video image frames from the front end in real time, performs accurate detection of human targets across the entire field of a single frame, and outputs the boundary information of the human target position; it associates and matches the detected human targets frame by frame, assigning a unique identifier to each human target; it completes continuous cross-frame trajectory association by combining the target's motion state, automatically assigns identifiers to newly added human targets in the field, and automatically cancels the corresponding identifiers of human targets that leave the field and disappear, so as to achieve uninterrupted and non-repeating tracking and management of human targets throughout the field.

4. The computer vision-based exhibition exhibit attention recognition system according to claim 1, characterized in that: The calculation of head pose and the completion of global coordinate system calibration, along with real-time calculation of spatial depth information and the degree of mutual occlusion between the human body, include: The system extracts features of the audience's head region within the image, resolves real-time head pose parameters by regression, and transforms the head pose parameters from the camera coordinate system to the global spatial coordinate system to complete global pose coordinate homogeneity calibration. It performs depth analysis on the image pixels frame by frame and obtains real physical space depth data by combining calibration parameters. It quantitatively calculates the overlapping occlusion area and occlusion ratio between multiple human targets by combining the position range of the human target with the depth hierarchy relationship between the front and back, and outputs the normalized occlusion degree value corresponding to each human target in real time.

5. The computer vision-based exhibition exhibit attention recognition system according to claim 1, characterized in that: The determination of on-site photo-taking actions, pointing actions, and mobile phone obstruction conditions includes: Based on the spatial relationship of key points of the human skeleton, the system verifies the hand raising height and local contour features in real time, and determines the on-site photo-taking action by combining the morphological features of electronic devices; based on the relationship between the human arm extension state and the relative height of the wrist, the system continuously verifies the continuity of limb posture and determines on-site pointing actions; the system detects the target contour shape of the area around the hand in real time, continuously verifies the presence of handheld devices for multiple frames, determines the situation of audience mobile phone occlusion, and simultaneously outputs binary status indicators of various actions and occlusion situations.

6. The computer vision-based exhibition exhibit attention recognition system according to claim 1, characterized in that: The verification and screening of the association between the human body and the exhibit space includes: The system retrieves the effective area and three-dimensional spatial location information of the preset exhibit image, combines the projection position of the human target image to determine the compliance of the area, and filters human targets within the imaging range of the exhibit. It also verifies the actual physical distance between the human body and the exhibit by combining global spatial depth data, eliminates invalid associated objects, and completes the two-layer spatial linkage verification and screening of human targets and corresponding exhibits.

7. The exhibition exhibit attention recognition system based on computer vision according to claim 1, characterized in that: The process of combining head orientation to complete the association matching and judgment includes: The system retrieves head orientation data of the audience under a unified coordinate system, combines it with the pre-set frontal spatial orientation information of the exhibits, and compares the degree of fit between the audience's actual gaze direction and the frontal orientation of the exhibits. It quantifies the head gaze deviation, classifies the correlation matching level according to the degree of gaze fit, eliminates invalid attention correlations due to gaze deviation, and outputs the effective correlation matching results between human targets and exhibits.

8. The computer vision-based exhibition exhibit attention recognition system according to claim 1, characterized in that: The execution of the single-person, single-exhibit exclusive optimal allocation binding includes: Establish the association weight relationship between human targets and exhibits, construct a two-way matching topology, and use a globally optimal matching strategy for unified calculation; set exclusive constraint rules that each person can only associate with a single exhibit at a time to avoid interference from repeated associations of multiple exhibits; complete priority matching and binding based on the association weight, filter out low-weight invalid association objects, and finally achieve one-to-one exclusive pairing and binding of human targets and exhibits throughout the venue without overlap.

9. The computer vision-based exhibition exhibit attention recognition system according to claim 1, characterized in that: The establishment of a full-time exhibition session timing management mechanism includes: The system monitors the binding status between human targets and exhibits in real time. When the binding status changes from invalid to valid, a new independent viewing session is automatically created. During the valid binding status, session operation data and behavior information are continuously recorded in time sequence. When the binding status is interrupted or an exhibit is switched, the current session is automatically closed and the session statistics are reset. The system manages the entire session lifecycle in a time-sharing manner, isolates viewing data from different time periods, and achieves uninterrupted time-series closed-loop management of sessions throughout the entire time period.

10. The computer vision-based exhibition exhibit attention recognition system according to claim 1, characterized in that: The comprehensive calculation of individual attention, combined with the score compensation correction based on on-site occlusion conditions, includes: Simultaneously collect posture stability data, effective dwell time, line-of-sight matching degree, and various interaction behavior data, and calculate the initial individual attention base score by multi-dimensional weighted fusion; retrieve on-site human occlusion data in real time to determine the strength of occlusion interference; and perform corresponding attenuation correction on the initial attention score according to the occlusion ratio to weaken the calculation deviation caused by severe occlusion and output the effective individual attention correction score.

Images