Portable psychosomatic assessment system based on deep learning eye movement and body posture tracking

Abstract

The application discloses a portable mind and body assessment system based on deep learning eye movement and body posture tracking, which comprises a flip type shell, a display, an upper camera, a lower eye movement tracking assembly, a microphone, a task response device and a computing unit. The camera collects body posture and facial images, and generates body key points and posture records from the posture images. Psychological behavior records are generated from eye movement, facial, voice, task selection, self-evaluation and behavior data. Cognitive task records are generated from the reaction time, accuracy and gaze path of attention, memory or execution control tasks. A deep learning inference module generates standardized cognitive task sub-scores according to eye movement and task characteristics. The three types of records share conversation and task identifiers, and retain time stamps and model versions. A quality gate module marks as limited or not computable when the source is missing or the input is invalid. The system encapsulates the three types of records and quality records into traceable data packets, and does not directly give medical or psychological diagnosis.

Description

Technical Field

[0001] This invention relates to a portable multimodal sensing and computer data processing device. The device acquires a user's posture images, eye movements, facial images, speech, and task responses within the same controlled task session. The device generates three types of intermediate records: body posture, psychological behavior, and cognitive task data. The device also utilizes a deep learning model to process eye movements and task features. The three types of records are encapsulated into a traceable data packet using a common identifier and quality status. Background Technology

[0002] Portable evaluation equipment requires maintaining the relative positions of the sensors after transport. Reinstalling the display, camera, and eye-tracking components before each use can easily lead to coordinate changes and recording mismatches. The top camera captures images of the head, shoulders, and upper body while the user is viewing the task. The eye-tracking component below the display captures gaze coordinates and gaze paths related to the on-screen task. Mounting both types of sensors within the same housing facilitates repeated deployment.

[0003] Postural, psychological, and cognitive task data are typically collected separately by different devices or programs. Different sources may use different sampling intervals and time bases. Merging data solely by timestamp makes it difficult to confirm whether records belong to the same task or trial. Traditional aggregation results may still provide unsubstantiated values ​​when a channel is missing or data is unreadable. Therefore, it is necessary to simultaneously save the common session identifier, task identifier, source timestamp, and quality status.

[0004] Eye movements and task performance can form multidimensional numerical features. Fixed rules are insufficient to express the non-linear relationships between features. Deep learning models can convert eye movement and task features into standardized cognitive task sub-scores after training, but if training labels, input computation, network layers, and model versions are not recorded, the results are difficult to reproduce. The model may also output seemingly valid values ​​when the input is missing or outside the valid range. Therefore, it is necessary to save the label source, feature parameters, model configuration, input validation, and quality gating along with the sensor records. Existing technology

[0005] Existing technologies have disclosed devices that house displays, cameras, or processors within transportable housings. Some devices also disclose eye-tracking, gesture, or user behavior tracking. Other systems present visual stimuli on a screen and evaluate them using gaze data, image data, or voice data. Relevant examples include US 2025 / 0322962 A1, WO 2013 / 012950A1, EP 2 590 047 A1, US 2020 / 0352499 A1, WO 2018 / 055618 A1, US 2020 / 0029806 A1, US 2016 / 0034141 A1, and US 2016 / 0022193 A1. The aforementioned documents relate to portable structures, sensors, or single-class evaluation data.

[0006] Upon review, the applicant has not found any document that fully discloses the combination of this application. This combination, within a fixed cover layout, includes an upper posture camera and a lower eye-tracking component, generating three types of identifiable records under the same task identifier. This combination also enables deep learning cognitive reasoning to be controlled by input validation and model versioning, and explicitly writes uncomputable states into the data packet. The above description is for illustrative purposes only and does not constitute an acknowledgment of the legal status of any document.

[0007] References

[0008] [1] US 2025 / 0322962 A1, Evaluation method of the level of fatigue of an operator and associated evaluation system.

[0009] [2] WO 2013 / 012950 A1, Integrated portable medical diagnostic system.

[0010] [3] EP 2 590 047 A1, Portable device.

[0011] [4] US 2020 / 0352499 A1, Adaptive psychological assessment tool.

[0012] [5] WO 2018 / 055618 A1, Screening apparatus and method.

[0013] [6] US 2020 / 0029806 A1, Eye-tracking system for detection of cognitiveload.

[0014] [7] US 2016 / 0034141 A1, Delay of display event based on user gaze.

[0015] [8] US 2016 / 0022193 A1, Real time biometric recording, information analytics and monitoring systems for behavioral health management. Summary of the Invention

[0016] To address the issue of unstable sensor relationships after portable deployment, this invention provides a flip-top housing with a base and a cover. A display is fixed inside the cover. A camera assembly is fixed above the display, facing the user. An eye-tracking assembly is fixed below the display. A computing unit, microphone, and task response device are connected to the aforementioned components. A session controller presents the task on the display and creates a session identifier, a task identifier, and an optional trial identifier.

[0017] The overhead camera captures timestamped image frames containing body posture and facial regions. The posture processor extracts body key points, contours, or motion features from the posture regions. The eye-tracking component acquires gaze coordinates, dwell events, and gaze paths. The microphone and task response device acquire voice, task selection, self-assessment, and behavioral responses. The controller generates a body posture record based on the posture images and features.

[0018] The controller generates psychological and behavioral records based on eye-tracking data, and writes one or more of the following to the record when available: facial images, speech, task selection, self-assessment, and behavioral responses. The controller also generates cognitive task records based on answers, accuracy, reaction time, and gaze paths for attentional, memory, or performance control tasks. All of these records are observable or intermediate evaluation data. The system does not directly determine disease names, psychological diagnoses, or IQ. The deep learning inference module receives ten normalized features arranged in a fixed order.

[0019] The first four terms are the maximum adjacent eye-tracking sampling interval and the dwell time for keywords, instructions, and background regions of interest. The last six terms are the recursion rate, determinism, and entropy of the one-dimensional and two-dimensional gaze trajectories, respectively. This module employs a ten-node input layer, a thirty-two-node first hidden layer, a sixteen-node second hidden layer, and a single-node output layer. The two hidden layers use ReLU activation with 25% random deactivation, and the output layer uses Sigmoid activation. The output is a standardized cognitive task sub-score in the range of zero to one, and is saved along with the task model identifier, model configuration identifier, and input reference.

[0020] The quality gating module checks the public identifier, required source fields, and numerical range. When the input meets the configured conditions, the corresponding dimension is marked as available. If only a non-required source is missing, the corresponding dimension is marked as restricted. If a required field is missing, unreadable data exists, or the deep learning input is invalid, the corresponding dimension is marked as uncomputable. When an input is uncomputable, the model output is not replaced with a default value or a fictitious value.

[0021] All three record types use the same applicable session identifier and task identifier. Each source retains its own source identifier, source timestamp, and status value. The controller stores the three record types, quality record, and model configuration identifier in a comprehensive evaluation data packet. The data packet allows authorized personnel to trace the source and model used for each result. This structure reduces cross-device record mismatches and improves verifiability after repeated deployments. Attached Figure Description

[0022] Figure 1 This is a front view of the portable psychophysical assessment system in its unfolded state. Figure 2 Functional block diagrams generated for multimodal acquisition, session control, and three types of recording. Figure 3 This is a functional block diagram for reasoning and comprehensive evaluation of data packages in deep learning cognitive tasks. The same reference numerals in each diagram represent the same or corresponding components. The abstract figures are designated as... Figure 1 .

[0023] Explanation of reference numerals in the attached figures

[0024] 400: Portable physical and mental health assessment system; 410: Flip-top housing; 411: Base; 412: Cover; 413: Hinge; 414: Locking assembly; 415: Handle.

[0025] 420: Display; 430: Top camera assembly; 431: Body posture area image data; 432: Facial area image data; 433: Body posture data processor; 440: Eye tracking assembly; 450: Computing unit; 460: Support insert; 480: Internal wiring area; 481: Microphone input; 484: Task response input device.

[0026] 490: Session controller; 491: Processor; 492: Memory; 493: Record manager; 495: Deep learning inference module; 496: Quality gating module.

[0027] 501: Body posture record; 502: Psychological and behavioral record; 503: Cognitive task record; 504: Quality record; 505: Comprehensive assessment data package; 506: Three-axis assessment overview; 507: Model configuration identifier.

[0028] Best practice and specific implementation

[0029] The following embodiments are based on the existing AMEND portable hardware prototype, its accompanying data acquisition and feature generation programs, and independently running quality gating and model selection reference programs. The hardware prototype and processing programs have been implemented separately; the quality gating and model selection reference programs have not yet been incorporated into the product release. This embodiment illustrates a feasible implementation where these components form a closed-loop combination within the same computing unit, and does not imply completion of end-to-end product performance verification or superior performance compared to existing technologies. Casing color, surface treatment, and branding do not limit the scope of protection. Task content can be configured according to authorized scenarios. All data acquisition and processing must be performed with applicable authorization and in compliance with personal information protection requirements.

[0030] 1. Portable casing

[0031] Figure 1 The system 400 shown includes a flip-top housing 410. The housing 410 has a base 411 and a cover 412. The base 411 and the cover 412 are connected by a hinge 413. The housing is movable between a closed transport state and an unfolded use state. A locking assembly 414 keeps the housing closed during transport. A handle 415 is used for carrying the housing.

[0032] The display 420, the upper camera assembly 430, and the eye-tracking assembly 440 are mounted on the user side of the cover 412. The computing unit 450 is placed on the support insert 460 of the base 411. Cables enter the base from the cover via an internal wiring area 480 adjacent to the hinge 413. This arrangement avoids cables crossing the central area of ​​the display. The computing unit can be removed for maintenance. Maintenance does not require removing the three main components inside the cover.

[0033] 2. Product View

[0034] Figure 1 The drawing is based on the implemented product shape and the relative positions of the components. The upper camera assembly 430 is located above the display 420. The eye-tracking assembly 440 is located below the display 420. All three components face the user. The upper camera's field of view includes at least the visible head and shoulder areas. Figure 1The leads in the diagram are only used to indicate components and do not represent signal paths.

[0035] 3. Fix sensor layout and coordinate data

[0036] The display 420 is fixed inside the cover 412. The upper camera assembly 430 and eye-tracking assembly 440 are fixed above and below the display, respectively. During normal transport and unfolding, the mounting areas of all three remain unchanged. The memory 492 stores the coordinate transformation from the eye-tracking assembly to the display coordinate system. The controller uses this transformation to convert eye-tracking samples into screen gaze coordinates. This fixed arrangement reduces the need to rearrange the sensors before each use.

[0037] Camera component 430 outputs an image frame identifier and a camera source timestamp. Eye-tracking component 440 outputs an eye-tracking source identifier and an eye-tracking source timestamp. Image coordinates and display coordinates are stored separately. The controller does not assume that the two sources share a hardware clock. Task identifiers are used to link related source items to the same task presentation. The internal order of each data stream is still determined by its source timestamp.

[0038] 4. Base computing unit and data flow

[0039] The computing unit 450 includes a processor 491 and a memory 492. A support insert 460 defines the mounting location of the computing unit. An internal wiring area 480 connects the cover sensor to the computing unit. The computing unit can be a laptop computer, a minicomputer, or a module with equivalent processing power. The session controller 490, the record manager 493, the deep learning inference module 495, and the quality gating module 496 are executed by the processor. The memory 492 stores task definitions, coordinate transformations, model files, model configuration identifiers, and quality configurations.

[0040] The memory also stores source and output records. The record manager 493 examines the field structure and establishes session, task, and trial indexes. The deep learning inference module 495 performs inference only when quality gating allows. After inference is complete, input references and model configuration identifiers are saved along with the output. This allows the processing path of a given output to be reconstructed.

[0041] 5. Body tracking based on the overhead camera

[0042] When a user views the display 420, the overhead camera assembly 430 captures a sequence of timestamped images. The images include at least two regions: the head, shoulder line, or upper body. Body region image data 431 and facial region image data 432 may originate from the same image frame. Both types of data retain the same source image frame identifier. The facial region may partially overlap with the body region. Whether the original images are permanently stored can be determined by privacy settings.

[0043] The posture data processor 433 detects body key points, body contours, or both in the posture region. Each detection result includes image coordinates, visibility status, and an optional detection confidence value. Positions can be normalized by image width and height. Corresponding points in consecutive frames are arranged by source timestamps. This allows for the acquisition of head tilt, shoulder line direction, upper body displacement, amplitude of movement, or duration of posture. The image frame identifiers used are written to the body posture record 501.

[0044] Body posture record 501 also stores session identifiers and task identifiers. A posture dimension can be marked as available or restricted when an image frame exists and at least one required posture feature is readable. If an image is missing, unreadable, or cannot be associated with a task, the dimension is marked as uncomputable. The record manager can still store the cause code for the missing feature. The system does not directly infer diseases from posture features. Posture records are used for subsequent non-diagnostic review.

[0045] 6. Psychological and behavioral records

[0046] The psychological and behavioral record 502 includes eye-tracking data and may include one or more of the following: facial images, speech, task selection, self-rating responses, and behavioral responses. Eye-tracking data may include gaze coordinates, gaze events, dwell events, and gaze paths. Facial image data 432 includes facial regions and their source timestamps. Microphone input 481 acquires speech items. Task response input device 484 acquires selection values, self-rating values, or action events. Each source item retains a source identifier and a status value.

[0047] Voice items can extract response latency, pause duration, speech rate, or volume variations. Facial regions can extract observable features such as motor units, the amount of facial expression changes, or head orientation. Task selection and self-assessment responses are saved by task identifier. Behavioral responses can include action events such as starting, pausing, withdrawing, or repeating selection. Features used maintain a reference relationship with the original source items. Psychological and behavioral records do not output disease names or psychological diagnoses.

[0048] In a minimum computable configuration, an eye-tracking source must be present, and at least one of the following sources must be present: facial, speech, task selection, self-assessment, or behavioral. A dimension is marked as available when the eye-tracking source and at least one other source are readable. A dimension can be marked as restricted when only an eye-tracking source is present or when a non-essential source is missing. A dimension is marked as uncomputable when an eye-tracking source or task identifier is missing. Quality record 504 stores the configuration used and the cause code. This decision does not replace missing sources with estimated data.

[0049] 7. Deep Learning Cognitive Task Inference

[0050] The controller displays attention, memory, or execution control tasks on the display 420. The task response input device 484 records the answer, accuracy, reaction time, and completion status. The eye-tracking component 440 records the gaze path during the task. The memory 492 stores the transformation from eye-tracking coordinates to display coordinates. The recording manager 493 uses this transformation to obtain the screen gaze coordinates and forms candidate cognitive task records according to the task identifier. The deep learning inference module 495 extracts ten features from the eye-tracking records and task records.

[0051] Each task configuration predefines keyword regions of interest (ROIs), instruction regions of interest (IPIs), and background regions of interest (RIs). The keyword IPI contains task-related target text or objects, the instruction IPI contains operation prompts, and the background IPI contains other non-target areas. The recording manager assigns screen gaze coordinates to a category in the order of instruction IPIs, keyword IPIs, and background IPIs. The first temporal feature is the maximum time interval between two adjacent valid eye-tracking samples within the session. The dwell time in each IPI category is the sum of the adjacent sampling time intervals corresponding to the valid eye-tracking sampling points assigned to that IPI; it is not required to pre-identify these sampling points as independent gaze events. If a task does not have a keyword IPI configured, the corresponding value is zero, and this fact is retained in the task configuration.

[0052] Gaze patterns are also used for recursive quantization analysis. The example generates a one-dimensional sequence of normalized screen horizontal coordinates and a two-dimensional sequence of normalized screen horizontal and vertical coordinates. Both sequences use an embedding dimension of 2 and a time delay of 1. Recursion is defined as a distance between two embedding vectors less than or equal to 0.055. The one-dimensional sequence uses the sum of the absolute differences of each dimension, and the two-dimensional sequence uses Euclidean distance. The minimum length of the recursion line is 2.

[0053] The recursion rate is the ratio of the number of recursive points to the total number of points in the recursion matrix. Determinism is the ratio of the number of recursive points contained in a diagonal continuous line of length at least 2 to the total number of recursive points. Entropy is calculated as the base-2 Shannon entropy based on the frequency of the length of the diagonal continuous line of length at least 2. The recursion matrix retains the main diagonal and does not use a Theiler exclusion window. These three metrics are calculated for both one-dimensional and two-dimensional sequences to form six recursive features. In a reproducible embodiment, the feature vector sequentially includes the maximum adjacent eye-tracking sampling interval, keyword interest region dwell time, instruction interest region dwell time, background interest region dwell time, one-dimensional recursion rate, one-dimensional determinism, one-dimensional entropy, two-dimensional recursion rate, two-dimensional determinism, and two-dimensional entropy.

[0054] When constructing the training data, the maximum adjacent eye-tracking interval was truncated to the 5th and 95th percentiles of the modeling data and then min-max normalized. The dwell times of the three interest regions were truncated to the 5th and 98th percentiles and then min-max normalized, respectively. The six recursive features were normalized to the minimum and maximum values ​​of the modeling data, respectively. The resulting normalized parameters were written into the model configuration along with the ten features in sequence. This configuration was used during deployment and inference, and the truncated or missing states were saved in the quality log. In this development data, the lower and upper normalization limits for the maximum adjacent eye-tracking interval were 0.123 and 0.97605, respectively.

[0055] The corresponding boundaries for dwell time in keywords, instructions, and background interest areas are 0 and 46.48518, 0.7273 and 28.84796, and 0.36795 and 14.31188, respectively. The boundaries for the three one-dimensional recursive features are 0.029815 and 0.286954, 0.531915 and 0.955556, and 0.887853 and 3.561459, respectively. The boundaries for the three two-dimensional recursive features are 0.009623 and 0.242213, 0.580827 and 0.965517, and 0.721928 and 3.820977, respectively. When different modeling data produce different boundaries, a new model configuration identifier should be created. This embodiment trains five regression models for five types of cognitive tasks. The five types of tasks correspond to time orientation, location orientation, immediate memory, continuous subtraction, and delayed recall, respectively.

[0056] Training labels are derived from the corresponding manual sub-items of the Mini-Mental State Examination. The highest scores for the five task categories are 5, 5, 3, 5, and 3, respectively. Each task sub-item is divided by its corresponding highest score to obtain a training target in the range of zero to one. The task identifier for each session is used to select the corresponding regression model. The model output is a standardized cognitive task sub-score estimate for the corresponding task. This score is only a computer-generated intermediate result for authorized personnel review and does not independently determine disease, mental state, or intelligence level.

[0057] The fully connected neural network has a 10-node input layer, a 32-node first hidden layer, a 16-node second hidden layer, and a single-node output layer. Both hidden layers use ReLU activation and 25% random deactivation. The output layer uses Sigmoid activation. The training loss is mean squared error. The optimizer is Adam, with an initial learning rate of 0.0003 and weight decay of 0.0005. The batch size is 32, and the maximum number of training epochs is 300.

[0058] The validation set comprises 30% of the dataset, and the learning process stops early if there is no improvement after 60 consecutive rounds. The learning rate decreases by a factor of 0.5 if the validation loss fails to improve for 8 consecutive rounds, with a minimum of 0.00001. The random seed is 42. The model package deployed to system 400 includes at least the weight file, the order of ten features, normalization parameters, network layers, task label meanings, and model configuration identifier 507. The inference vector carries the names or location identifiers of the ten features. The disease group to which the subject belongs is not used as input during inference.

[0059] The system first verifies the limited number and values ​​of features and checks that the names or positional identifiers match the fixed order in the model configuration. Values ​​between -0.000001 and zero are truncated to zero, and values ​​between one and 1.000001 are truncated to one, with the truncation status saved in the quality record. If any required feature is missing, the identifier order is mismatched, or the value falls outside the range of -0.000001 to 1.000001, the quality gating module 496 blocks inference. When inference is blocked, the cognitive task record 503 is marked as uncomputable and a reason code is saved. When inference is successful, the record saves the task type, task performance, gaze path reference, and standardized cognitive task sub-score.

[0060] 8. Identifier association, quality gating, and data packets

[0061] Session controller 490 creates a session identifier for each evaluation. Each task presentation has a task identifier. When a task contains multiple trials, each trial also has a trial identifier. Body posture record 501, psychological behavior record 502, and cognitive task record 503 use the same applicable identifier. Each source item separately stores a source identifier, timestamp, format identifier, and status value. Common identifiers are used for cross-source association, and source timestamps are used to maintain the internal order of each data stream.

[0062] The Quality Gating Module 496 first checks the session identifier and task identifier. It then checks the required sources and fields specified for the selected dimension. For deep learning inputs, it also checks the number of features, feature names or location identifiers provided with the vector, numerical finiteness, and normalization range. Names or location identifiers should conform to a fixed order in the model configuration. Slight out-of-tolerance values ​​within the tolerance range are truncated to a range of zero to one; values ​​exceeding the tolerance prevent inference. The status is available when all required conditions are met.

[0063] The status is restricted only when a non-essential source is missing. The status is uncomputable when the essential condition is not met. Each judgment saves the quality configuration identifier and reason code. The quality record 504 includes the presence value of each channel, the readable value of the source, the complete value of the identifier, and the computable status of the dimension. The reason code can indicate that the identifier is missing, the image is unreadable, the eye movement is missing, the response is incomplete, the feature is out of range, or the model is unavailable.

[0064] When computation is not possible, the system retains the source records already acquired. The system does not generate alternative deep learning metrics. Authorized personnel can use this information to decide whether to re-collect data. This mechanism clearly distinguishes failed states from normal values. The record manager 493 writes the three types of records and the quality record 504 into the comprehensive evaluation data package 505.

[0065] The data package also stores the task definition, coordinate transformation version, and model configuration identifier 507. The optional three-axis evaluation profile 506 only summarizes intermediate metrics whose status is either available or restricted. Each value in the profile still references its source record. Uncomputable dimensions remain null in the profile and carry a reason code. The data package can be encrypted and stored locally or transmitted to an authorized review system.

[0066] 9. Operating Procedures

[0067] The operator opens the flip-top housing 410 and confirms that the user is within the field of view of the upper camera assembly 430. The controller loads the task, coordinate transformation, model package, and quality configuration. The controller creates a session identifier. The display 420 presents the task with the task identifier. The camera, eye-tracking assembly, microphone, and task response device begin acquisition.

[0068] Each source item is recorded with a source timestamp and status value. The posture data processor 433 generates body key points and posture features. The record manager 493 forms a body posture record 501 and a psychological behavior record 502. Upon completion of the task, the record manager forms a ten-item cognitive task feature vector with feature names or location identifiers. The quality gating module 496 checks the number of vectors, the order of identifiers, the finiteness of values, and the range of values.

[0069] Slight out-of-tolerance values ​​within the tolerance range are truncated to the range of zero to one. If the check passes, the deep learning inference module 495 generates a standardized cognitive task sub-score. If the check fails, the system records an incalculable state and does not perform inference. The controller then generates a quality record 504. The three types of records and the quality record are packaged into a comprehensive evaluation data package 505.

[0070] The data package stores the model configuration identifier and source reference. The display interface can show intermediate results and their quality status across three dimensions: physical posture, psychological behavior, and cognitive tasks. The display interface also indicates that the results are used to assist in review. The system does not automatically output treatment suggestions.

[0071] 10. Deformation methods and technical effects

[0072] The overhead camera can be a visible light camera, a depth camera, or a combination of cameras. The posture data processor can employ keypoint detection, contour detection, or motion tracking. The source of psychological behavior can be selected based on the scene, but at least eye-tracking sources must be retained. The five cognitive tasks can change the presentation of text, objects, or response methods, but the meaning of the task labels and the highest score should remain unchanged. The same 10-32-16-1 network can train weights separately for different tasks. Each set of weights and normalization parameters should use different model configuration identifiers.

[0073] A fixed sensor layout allows the system to maintain the relative positions of the camera, display, and eye-tracking components after transport. Common identifiers reduce erroneous associations between different sources. Quality gating prevents missing or out-of-bounds inputs from being interpreted as normal model results. Model version and source citations improve the reproducibility of results. The three types of records can be read separately and jointly verified within the same data packet. These technical effects result from the combined effect of hardware layout, data association, and controlled inference.

[0074] Industrial applicability

[0075] This system can be manufactured using existing housings, displays, cameras, eye trackers, microphones, input devices, and computers. The program can run on a general-purpose processor or graphics processor. The system is suitable for authorized schools, research institutions, training facilities, community services, and workplaces. Results of physical posture, psychological behavior, and cognitive tasks are output as intermediate records. The system can be repeatedly deployed and generates auditable data packets. Therefore, this invention has industrial applicability.

Claims

1. A portable psychophysical assessment system (400) based on deep learning eye-tracking and body posture tracking, comprising: A flip-top housing (410) having a base (411) and a cover (412). A display (420) fixed to the cover (412); a camera assembly (430) fixed above the display (420), the field of view of the camera assembly (430) including the user's body posture area and face area; an eye-tracking assembly (440) fixed below the display (420); a microphone input (481); a task response input device (484); a computing unit (450); and a session controller (490), a body posture data processor (433), a recording manager (493), a deep learning inference module (495), and a quality gating module (496) executed by the computing unit (450); wherein the session controller (490) is configured to create session identifiers and task identifiers and present cognitive tasks on the display (420); the body posture data processor (433) is configured to generate body posture features from body posture area image data (431) obtained from the camera assembly (430); the recording manager (493) The system is configured to generate a body posture record (501) based on the body posture region image data (431) and the body posture features, generate a psychological behavior record (502) based on the eye movement data obtained by the eye movement tracking component (440), and write one or more of facial images, speech, task selection, self-evaluation response and behavioral response into the psychological behavior record (502) when available, and allocate eye movement samples to keyword interest areas, instruction interest areas or background interest areas using the transformation from stored eye movement coordinates to display coordinates; the record manager (493) is also configured to form ten normalized features with feature names or location identifiers in a fixed order, the ten normalized features being, in order, the maximum adjacent eye movement sampling interval, keyword interest area dwell time, instruction interest area dwell time, background interest area dwell time, one-dimensional recursion rate, one-dimensional determinism, one-dimensional entropy, two-dimensional recursion rate, two-dimensional determinism and two-dimensional entropy, wherein the one-dimensional and two-dimensional recursive features adopt an embedding dimension of 2, a time delay of 1, and a recursion threshold of 0.055 and minimum recursion line length 2 are calculated; the deep learning inference module (495) sequentially includes a ten-node input layer, a thirty-two-node first hidden layer with ReLU activation and twenty-five percent random deactivation, a sixteen-node second hidden layer with ReLU activation and twenty-five percent random deactivation, and a single-node output layer with Sigmoid activation, and is configured to select the corresponding task regression model according to the task identifier, RQA parameter signature and model configuration identifier (507), and generate a standardized cognitive task sub-score written to the cognitive task record (503) according to the ten normalized features; the quality gating module (496) is configured to select the corresponding task regression model according to the corresponding record. The necessary source and public identifiers determine the availability, limitation, or incomputability of the respective body posture records (501), psychological behavior records (502), and cognitive task records (503), and check the quantity of the ten normalized features, the consistency of the feature names or location identifiers with the fixed order, and the validity of the values ​​before executing the deep learning inference module (495); and the record manager (493) is configured to allow the three types of records to share applicable session identifiers and task identifiers, while retaining the source timestamp and model configuration identifier (507), and storing the three types of records along with the quality record (504) in a comprehensive evaluation data package (505).

2. The system (400) according to claim 1, wherein, The housing (410) is movable between a closed transport state and an unfolded use state; the display (420), the camera assembly (430), and the eye-tracking assembly (440) are located in the fixed mounting area of ​​the cover (412); the computing unit (450) is detachably supported in the support insert (460) in the base (411); and the cable connecting the cover (412) and the computing unit (450) passes through the internal wiring area (480) adjacent to the hinge (413).

3. The system (400) according to claim 1, wherein the body posture data processor (433) is configured to detect body key points, body contours or both in body posture region image data (431), and generate at least two of head tilt, shoulder line direction, upper body displacement, movement amplitude and posture duration based on one or more timestamped image frames; the body posture record (501) stores references to the image frames used.

4. The system (400) according to claim 1, wherein, The mental behavior record (502) stores a reference to each source item used; when an eye-tracking source and at least one other source among facial images, speech, task selection, self-rating response and behavioral response are present, the quality gating module (496) marks the mental behavior record (502) as available; when only an eye-tracking source is available, it is marked as restricted; when an eye-tracking source or task identifier is missing, it is marked as uncomputable.

5. The system (400) according to claim 1, wherein, The labels used to train the deep learning reasoning module (495) include artificial sub-items for five types of tasks: time orientation, location orientation, immediate memory, continuous subtraction, and delayed recall. The time orientation and location orientation sub-items are divided by 5, the immediate memory sub-item is divided by 3, the continuous subtraction sub-item is divided by 5, and the delayed recall sub-item is divided by 3 to form standardized cognitive task sub-scores in the range of zero to one. The standardized cognitive task sub-scores serve as intermediate results for authorized personnel to review, rather than directly determining the disease, psychological state, or intelligence level.

6. The system (400) according to claim 1, wherein, Each source item includes the session identifier, the task identifier, the source identifier, the source timestamp, the data format identifier, and the source status value; when the task includes multiple trials, it also includes the trial identifier; and the association of different source items does not require the sources to share a hardware clock.

7. The system (400) according to claim 1, wherein, The quality gating module (496) is configured to check the feature count of the ten normalized features, the consistency of the feature names or position identifiers provided with the vector with the fixed order in the model configuration, and the finiteness of the values. Values ​​between -0.000001 and zero are truncated to zero, and values ​​between one and 1.000001 are truncated to one. The truncation state is saved in the quality record (504). If any required feature is missing, identifier order is mismatched, or the value falls outside the range of -0.000001 to 1.000001, the deep learning inference module (495) is prevented from performing inference, and an uncomputable state and cause code are written.

8. The system (400) according to claim 1, wherein, The model configuration identifier (507) corresponds to the model weights, ten feature order, normalization parameters, network level, task label meaning, and output meaning of each of the five task regression models; the cognitive task record (503) stores the task model identifier, the model configuration identifier (507), RQA parameter signature, the transformation version from eye-tracking coordinates to display coordinates, and the source record referenced by the ten normalization features, so that the standardized cognitive task sub-scores can be verified.

9. A data processing method for a portable psychophysical assessment system, comprising: A cognitive task is presented on a display (420) fixed to the cover (412) of the flip-top housing (410), and a session identifier and a task identifier are created; a time-stamped image frame containing a body posture region and a facial region is received via a camera assembly (430) fixed above the display (420), and eye-tracking data is received via an eye-tracking assembly (440) fixed below the display (420), and a voice or task response is received; body posture features are generated from the image data of the body posture region, and a body posture record (501) is formed; a mental image is formed based on the eye-tracking data. The system records psychological behavior (502) and, when available, writes one or more of the following into the psychological behavior record (502): facial image, voice, task selection, self-evaluation response, and behavioral response; it uses the transformation from stored eye movement coordinates to display coordinates to assign eye movement samples to keyword interest areas, instruction interest areas, or background interest areas; it forms ten normalized features in a fixed order, including the maximum adjacent eye movement sampling interval with feature names or location identifiers, the dwell time of the three types of interest areas, and the recursion rate, determinism, and entropy of the one-dimensional and two-dimensional gaze trajectories, where the recursive features adopt an embedding dimension of 2 and a time delay of 1. The recursion threshold of 0.055 and the minimum recursion line length of 2 are calculated. The system checks the necessary sources, public identifiers, the number of the ten normalized features, the consistency of the feature names or positional identifiers with the fixed order in the model configuration, and the finiteness of values. Values ​​between -0.000001 and zero are truncated to zero, and values ​​between one and 1.000001 are truncated to one. The check fails if any necessary feature is missing, the identifier order is mismatched, or the value falls outside the range of -0.000001 to 1.000001. If the check passes, the task target is determined accordingly. The identifier, RQA parameter signature, and model configuration identifier (507) are used to select the task regression model, and the ten normalized features are sequentially input into a ten-node input layer, a thirty-two-node first hidden layer with ReLU activation and twenty-five percent random deactivation, a sixteen-node second hidden layer with ReLU activation and twenty-five percent random deactivation, and a single-node output layer with Sigmoid activation to generate standardized cognitive task sub-scores and write them into the cognitive task record (503); if the check fails, the affected record is marked as restricted or uncomputable and no alternative sub-scores are generated; The three types of records share the applicable session identifier and task identifier, while retaining the source timestamp and model configuration identifier (507); and the three types of records are stored in the comprehensive evaluation data packet (505) along with the quality record (504).

10. A non-transitory computer-readable storage medium having a computer program stored thereon, the computer program causing the processor, when executed by a processor of a portable psychophysical assessment system (400), to perform the data processing method of claim 9.

Images