Intelligent glasses for wounded person emotion recognition in complex first-aid environment

Abstract

The application relates to the technical field of data processing, in particular to intelligent glasses for wounded person emotion recognition in a complex emergency environment, which comprises a face data capturing module, a region trajectory extraction module, a variation interval marking module, a face response focusing module and a state expression output module.In the application, a mapping set is constructed according to image sequence numbers and thermal time stamps, time synchronization and structure interconnection between heterogeneous data are realized, continuous trajectories of a face feature region are extracted by fusing edge contour changes, stable trajectory segments are constructed by filtering mutation points, response state sequences are constructed through trajectory overlapping relationships between face key regions by combining the index and time interval difference of the trajectories in the image frames, the classification integration and number interlaced analysis of region linkage features are realized, the corresponding structure between the number combination and the expression label is established, the emotion state is transformed from one-way matching to multi-dimensional fusion, and the clarity and accuracy in time sequence expression and difference identification are improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a smart glasses for recognizing the emotions of injured persons in complex emergency situations. Background Technology

[0002] The field of data processing technology encompasses the entire process of information collection, storage, analysis, identification, and utilization. It covers core aspects such as classifying, modeling, mining, and transforming various structured and unstructured data using computers or intelligent devices, serving as the foundation for intelligent decision-making, system control, human-computer interaction, and big data analysis. This technology is widely applied in scenarios such as image recognition, speech processing, behavior analysis, and physiological signal recognition, and is gradually integrating cutting-edge interdisciplinary fields such as artificial intelligence, wearable devices, intelligent sensing, and emotion recognition. The aim is to improve processing efficiency and application accuracy through algorithmic rules, feature extraction, and multi-dimensional information fusion. Among these, traditional smart glasses used for emotion recognition in complex emergency situations refer to intelligent recognition devices designed for complex emergency rescue scenes such as public emergencies and disasters. These devices use wearable technology to perceive and make preliminary judgments on multimodal information such as facial expressions, voice tone, and physiological parameters of the injured, in order to identify their emotional state and assist medical personnel in quickly understanding their injuries and psychological state. These smart glasses typically collect image data through a built-in camera, acquire audio features through a voice acquisition module, and collect physiological indicators through temperature sensors, conductivity sensors, and other means. They then use facial expression matching rules, voice feature analysis, and physiological signal threshold determination to identify the type of emotion of the injured person, and finally transmit the recognition results to the medical staff's terminal in the form of images, voice, or text.

[0003] Existing technologies lack a unified sorting mechanism in the processing of image and physiological information, which can easily lead to weak data matching and correlation. The time record of thermal information and the image frame number cannot be accurately matched, resulting in the risk of information disconnection and mixed interference. The region feature extraction does not introduce the continuous judgment of trajectory changes between frames, which can easily lead to the incomplete reflection of short-term dynamic changes. Anomalies in trajectory data are not effectively cleaned, affecting feature stability and expression consistency. The lack of a time-based linkage analysis structure between multiple facial trajectories causes the role of subtle parts in the interaction to be ignored. The label classification in the output results is not refined enough, making it difficult to form a clear judgment logic of emotional state, thus reducing the reference value and practicality of the recognition results. Summary of the Invention

[0004] To address the technical problems existing in the prior art, embodiments of the present invention provide smart glasses for recognizing the emotions of injured persons in complex emergency rescue environments. The system includes:

[0005] The facial data capture module collects images and thermal information acquired by the smart glasses' camera and temperature sensing area during operation, reads the image frame number and thermal timestamp, removes data without number and data with discontinuous time, and generates frame-encoded image sequence groups in chronological order. The region trajectory extraction module, based on the frame-encoded image sequence group, marks the facial region, extracts the boundary trajectories of eyebrows, eyes, nose wings, and corners of the mouth, calculates the coordinate difference between the trajectory points in the preceding and following frames, retains continuous trajectory points, and generates a set of region trajectory fragments. The variable interval marking module extracts the region trajectory set fragments, records the frame index and number, constructs the inter-frame change structure, identifies trajectory combinations that are continuous in time and whose coordinate changes exceed a preset threshold, and generates a trajectory interval combination identifier set. The facial response focusing module classifies trajectories according to facial position based on the trajectory interval combination identifier set, counts the frequency, judges the number overlap, retains continuous interlaced combinations, and generates a sequence of overlapping relationship of regional arrangement; The state expression output module extracts the number combination of the overlapping relationship sequence of the regions, maps it to the facial region label, encapsulates the content of the label, and inputs it into the output channel to obtain the injury patient's emotion recognition result.

[0006] As a further aspect of the present invention, the frame-encoded image sequence group includes image frame sequence number, corresponding thermal timestamp, and mapping relationship between image and thermal information; the region trajectory set fragment includes facial region boundary line, feature part trajectory outline, and continuous inter-frame coordinate sequence; the trajectory interval combination identifier set includes trajectory start and end frame number, inter-trajectory time interval, and trajectory number mapping list; the region arrangement overlap relationship sequence includes trajectory category grouping, number of intra-frame trajectories, and overlap number combination; and the injured person emotion recognition result includes number combination matching record, expression label name, and label encapsulation output content.

[0007] As a further aspect of the present invention, the combination of temporally continuous and differentiated trajectories refers to a set of facial region trajectories that have coordinate differences in adjacent frames but are temporally continuous.

[0008] As a further aspect of the present invention, the continuous interleaved combination refers to a combination of regions whose numbers appear alternately in time and whose trajectories are less than a set continuity threshold in space.

[0009] As a further aspect of the present invention, the facial data capture module includes: The synchronous calibration submodule is based on the image data set and thermal information record set captured by the camera area and temperature sensor embedded in the structure of the smart glasses during the working period. It obtains the image frame sequence number and thermal information timestamp content, determines whether the number and timestamp are missing or misaligned, removes the corresponding abnormal data frames and time records, and generates a synchronous index removal sequence. The image-thermal mapping submodule calls the synchronous index removal sequence to retain the index, filters and arranges the image data groups and thermal information in chronological order, establishes a key-value correspondence between each group of image frames and thermal parameters, and generates an image-thermal information mapping set. The frame sequence generation submodule extracts the image frame number and re-encodes it according to the time sequence structure in the image heat map set. The corresponding frame image data are arranged in order to generate a frame-encoded image sequence group.

[0010] As a further aspect of the present invention, the region trajectory extraction module includes: The boundary trajectory extraction submodule, based on the frame-encoded image sequence group, obtains the sub-image content marked as the facial region in each frame, identifies edge contour lines, separates the boundary lines of eyebrows, eyes, nose wings, and corners of the mouth, extracts the trajectory point coordinates of the region and removes breakpoints, and generates a set of region boundary trajectories. The trajectory continuity submodule calls the coordinates of trajectory points in the trajectory set of the region boundary, calculates the coordinate difference of each point between adjacent frames, compares it with the jump logic reference value, determines that points with a value greater than the jump logic reference value are abnormal jump points and removes them, and aggregates the remaining coordinate points according to the frame order to generate a continuous trajectory point sequence. The regional trajectory construction submodule integrates the point sequences of the same region into continuous data packets according to time based on the continuous trajectory point sequence, performs trajectory segment connection processing, constructs the trajectory structure, and generates regional trajectory set fragments.

[0011] As a further aspect of the present invention, the variable interval marking module includes: The frame sequence numbering submodule reads the position index of the trajectory line segment in the image frame sequence based on the region trajectory set fragment, records the occurrence number of the point in each trajectory in the frame sequence, constructs the inter-frame continuous structure according to the numbering order, extracts the start and end frame numbers of the trajectory segment, and generates the trajectory frame sequence number set. The interval filtering submodule calls the start and end frame numbers in the trajectory frame sequence number set, calculates the time interval sequence between the first and last frames for each trajectory, judges the trend of time interval change, filters the trajectory combination with continuous time interval and coordinate change exceeding the preset threshold, and generates a continuously changing trajectory group. The interval identifier generation submodule extracts the number information of the trajectories in the combination based on the continuously changing trajectory group, classifies all the numbers and establishes a number mapping relationship table, encodes the mapped numbers into the index list after dividing them into combinations, and generates a trajectory interval combination identifier set.

[0012] As a further aspect of the present invention, the facial response focusing module includes: The trajectory classification submodule extracts the image position index corresponding to each trajectory number based on the trajectory interval combination identifier set, classifies the trajectories according to the facial structure coordinate system, divides all numbers into four categories: nose wing, eyebrow, corner of mouth, and eye, accumulates the number of times the trajectory appears in the image frame sequence, and generates a regional trajectory classification statistics table. The interlacing determination submodule calls the trajectory number and frame index information in the regional trajectory classification statistics table, retrieves the position number of any two types of trajectories in adjacent frames, performs a range matching operation on the number interval, determines whether there is number overlap, and generates a list of trajectory interlacing numbers. The overlapping relationship submodule, based on the trajectory interlacing number list, filters out number combinations with continuous interlacing relationships, encodes and organizes trajectory sequences that meet the set continuous conditions, establishes a classification and combination sequence structure, sorts the interlacing sequences according to the starting frame index and writes them into the mapping structure set, and generates a region arrangement overlapping relationship sequence.

[0013] As a further aspect of the present invention, the state expression output module includes: The tag matching submodule extracts trajectory identification information from the numbered combinations based on the overlapping relationship sequence of the regions, calls the expression tag mapping directory registered in the smart glasses, matches the numbered combinations with the corresponding tag numbers in the directory, performs index comparison for each numbered combination, records the tag number results, and generates a tag number correspondence table. The tag encapsulation submodule calls the tag number results recorded in the tag number correspondence table, performs duplicate number deduplication and extracts all differentiated tag numbers, integrates the tag content associated with the tag number in turn, merges and encapsulates all content, and generates a tag encapsulation content set. The emotion output submodule encapsulates the content set based on the tags, imports all the encapsulated tag content into the result output channel, aggregates and encapsulates the content structure and labels it with numbers, and then outputs it to the recognition structure to generate the emotion recognition result of the injured person.

[0014] As a further aspect of the present invention, the step of deduplicating duplicate numbers and extracting all differentiated label numbers refers to extracting a set of all numbers that have uniqueness and distinguishability in the label numbers, based on excluding duplicate numbers.

[0015] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: Based on image sequence numbers and thermal timestamps, a mapping set is constructed to achieve time synchronization and structural connectivity between heterogeneous data. Continuous trajectories of facial feature regions are extracted by fusing edge contour changes, and stable trajectory segments are constructed by filtering abrupt changes. Combining the sequential index and time interval differences of the trajectories in image frames, a response state sequence is constructed through the overlapping relationship of trajectories between key facial regions. This enables the classification and integration of regional linkage features and the analysis of numbered interleaving, establishing a correspondence structure between number combinations and expression labels. This promotes the transformation of emotional states from one-way matching to multi-dimensional fusion, improving the clarity and accuracy in temporal expression and difference identification. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 This is a schematic diagram of the process framework of the present invention; Figure 3 This is a flowchart of the facial data capture module in this invention; Figure 4 This is a flowchart of the region trajectory extraction module in this invention; Figure 5 This is a flowchart of the variable interval marking module in this invention; Figure 6 This is a flowchart of the facial response focusing module in this invention; Figure 7 This is a flowchart of the state expression output module in this invention. Detailed Implementation

[0018] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0019] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0020] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent.

[0021] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0022] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0023] This invention provides a smart glasses system for recognizing the emotions of injured persons in complex emergency situations, such as... Figure 1-2 The diagram shown illustrates a smart glasses system for recognizing injured persons' emotions in complex emergency situations. The system includes: The facial data capture module collects image data sets and thermal information records captured by the camera area and temperature sensor embedded in the structure of the smart glasses during the working period. It reads the sequential number of the image frame sequence, organizes the timestamp content corresponding to the thermal information, judges whether there are missing or misaligned segments, removes data without number and discontinuous time, merges the images and thermal information and arranges them in chronological order to establish a mapping set, and generates frame-encoded image sequence groups. The region trajectory extraction module reads the frame-encoded image sequence group, marks the sub-image content of the facial region, identifies the edge contour lines in the facial image, separates the boundary trajectories of the eyebrow, eye, nose, and mouth regions, calculates the coordinate difference of each trajectory point in the previous and next frames, determines whether the coordinate difference is within the logical range of continuous jumps, removes the trajectory segments corresponding to the coordinate movement change points, integrates the remaining trajectory points to construct a continuous data packet, and generates a set of region trajectory fragments. The variable interval marking module calls the trajectory line segments in the region trajectory set fragment, reads the position index of each line segment in the image frame sequence, records the order of appearance of points in the continuous trajectory, constructs the inter-frame change sequence structure based on the point number, judges the time interval change trend between the start and end frames of the trajectory, extracts the trajectory combination with no time interval and coordinate change exceeding the preset threshold, summarizes the combination trajectory numbers into a set of mapping lists, and generates a trajectory interval combination identifier set. The facial response focusing module reads each group of trajectory identifiers from the trajectory interval combination identifier set, classifies them according to the facial position corresponding to the trajectory in the image, and classifies them into four types of trajectory sequences: nose wing, eyebrow, corner of mouth, and eye. It counts the number of times each type of trajectory appears in the image frame, performs interleaving judgment on the overlapping intervals of position numbers in adjacent trajectory sequences, retains the number combinations with continuous interleaving relationships, and generates a sequence of overlapping relationships of regions. The state expression output module extracts the number combinations from the overlapping relationship sequence of the region arrangement, calls the registered expression tag mapping directory in the smart glasses, establishes a corresponding record table between the number combinations and expression tags, queries the tag number corresponding to each number combination, merges the query results to obtain the tag encapsulation content, and imports the tag encapsulation results into the result output channel to generate the wounded person's emotion recognition result.

[0024] The frame-encoded image sequence group includes the image frame sequence number, the corresponding thermal timestamp, and the mapping relationship between the image and thermal information. The region trajectory set fragment includes the facial region boundary line, the trajectory outline of the feature part, and the continuous inter-frame coordinate sequence. The trajectory interval combination identifier set includes the trajectory start and end frame number, the time interval between trajectories, and the trajectory number mapping list. The region arrangement and overlap relationship sequence includes trajectory category grouping, the number of trajectories within the frame, and the overlap number combination. The patient emotion recognition result includes the number combination matching record, the expression label name, and the label encapsulation output content.

[0025] Specifically, such as Figure 2 , 3 As shown, the facial data capture module includes: The synchronous calibration submodule is based on the image data set and thermal information record set captured by the camera area and temperature sensor embedded in the structure of the smart glasses during the working period. It obtains the image frame sequence number and thermal information timestamp content, determines whether the number and timestamp are missing or misaligned, removes the corresponding abnormal data frames and time records, and generates a synchronous index removal sequence. First, based on the differences in the installation angles and positions of the camera module and the thermal sensing module on the smart glasses, the overlapping area of ​​their sensing regions is determined. Through internal device structural parameter settings, the horizontal viewing angle of the camera region is set to 120 degrees, the vertical viewing angle to 70 degrees, and the thermal sensing region's coverage angle to 95 degrees. A two-dimensional coordinate mapping relationship is established for the overlapping area of ​​these two regions in space. The effective interface range between the image frames corresponding to the overlapping area and the thermal sensing information is calculated. Then, a data acquisition task is initiated, acquiring the video stream collected by the camera module during the operation period and segmenting it into single-frame images. Each frame is assigned a unique number at a standard acquisition rate of 25 frames per second, with the numbers sequentially incremented starting from 0. Simultaneously, the temperature sensing module records thermal parameter data 5 times per second. Each thermal data point is accompanied by a timestamp, accurate to milliseconds. The timestamp is calculated from the Unix time base at the start of the task; for example, if the task start time is 1600000000000 milliseconds, the timestamp for the first thermal data point is... The time stamp is 1600000000200 milliseconds. Then, two time series are generated, one for the image frame number and the other for the thermal recording timestamp. Next, the frame numbers and timestamps are matched and verified to determine if there is a time interval greater than 40 milliseconds between any two frames or a thermal recording interval greater than 250 milliseconds. If the time for image frame number 36 is 1600000001440 milliseconds, and the time for the next frame number 37 is 1600000001520 milliseconds, the interval is 80 milliseconds, exceeding the set limit. Therefore, image frame number 37 is considered an abnormal image frame and is removed. If a thermal recording time is 1600000002980 milliseconds, and the next is 1600000003250 milliseconds, the interval is 270 milliseconds, it is also marked as an abnormal record and removed from the record set. The remaining successfully matched numbers and timestamps are then written into the synchronization index removal sequence, which is used for subsequent data filtering and mapping.

[0026] The image-thermal mapping submodule calls the synchronous index to remove the retained index in the sequence, filters and arranges the image data groups and thermal information in chronological order, establishes a key-value correspondence between each group of image frames and thermal parameters, and generates an image-thermal information mapping set. First, the synchronization index removal sequence is traversed, reading each pair of image frame numbers and their corresponding timestamps. Only the image corresponding to that number is retained in the image data group. Thermal sensing records with an error of no more than ±20 milliseconds from the timestamp are selected from the thermal sensing record set. For example, if image frame number 55 corresponds to time 1600000009876 milliseconds, the system will search the thermal sensing record set for thermal sensing records within the interval from 1600000009856 to 1600000009896 milliseconds. If a thermal sensing record is found with a time of 1600000009881 milliseconds, this record is bound to frame number 55. Then, all frame numbers are sorted in ascending order according to their timestamps, and the images are recombined with their corresponding thermal sensing data based on the sorting results. During the process of establishing the mapping relationship, the thermal sensing value, ambient temperature, and other parameters in the thermal sensing information record are read one by one. The temperature difference corresponding to the current frame image is calculated and recorded as a mapping item in the key value corresponding to the frame number. For example, the thermal sensing value corresponding to frame number 55 is 36.9 degrees Celsius and the ambient temperature is 25.2 degrees Celsius, so the temperature difference is 11.7 degrees Celsius. This value is directly appended to the image record corresponding to number 55. The above operation is repeated until all the numbers in the synchronization index have completed the filtering, matching, and mapping tasks. Finally, the image frame number and its corresponding thermal sensing value, ambient temperature, and temperature difference are combined into a key-value pair set, and all data in the set are arranged in ascending order of timestamp. This mapping set will serve as the basic data structure for the next step of image sequence generation.

[0027] The frame sequence generation submodule extracts the image frame number and re-encodes it according to the time sequence structure in the image heat information mapping set. The corresponding frame image data are arranged in order to generate a frame-encoded image sequence group. First, all key-value pairs in the mapping set are read, the timestamps corresponding to all image frame numbers are identified, and these timestamps are sorted in ascending order. The image frame numbers are then rearranged according to the sorted time order, and a mapping table corresponding to the old and new numbers is established. The frame number corresponding to the earliest timestamp is redefined as 0, and the sequential number is incremented by 1 for each additional frame. For example, the timestamps of the original frame numbers 45, 50, 52, and 57 are 1600000000450, 1600000000650, 1600000000730, and 1600000000890 milliseconds, respectively. After sorting, they are assigned new numbers 0, 1, 2, and 3, and the corresponding image frames are renamed frame_000.jpg, frame_00, etc., respectively. The image frames 1.jpg, frame_002.jpg, and frame_003.jpg are then arranged and stored in the file system according to the new numbering order. At the same time, the corresponding thermal parameter information is stored in the corresponding information file of the image frame in text or structured data form. For example, the information file corresponding to frame_000.jpg stores parameter values ​​such as thermal value of 37.2 degrees Celsius, ambient temperature of 25.0 degrees Celsius, and temperature difference of 12.2 degrees Celsius. The entire renaming, rearrangement, and information addition operation is performed item by item for each image frame until all image frames are recoded, generating an image sequence group with continuous frame numbering, consistent order, and temperature information, which is used for subsequent analysis or display processing.

[0028] Specifically, such as Figure 2 , 4 As shown, the region trajectory extraction module includes: The boundary trajectory extraction submodule is based on frame-encoded image sequence groups, obtains the sub-image content marked as facial regions in each frame, identifies edge contour lines, separates the boundary lines of eyebrows, eyes, nose wings, and corners of mouth, extracts the trajectory point coordinates of the region and removes break points, and generates a set of region boundary trajectories; First, each frame of the image is read sequentially, and the sub-image portion marked as the facial region is identified. This marking is used to extract the content of the local facial region image through the rectangular coordinate information in the image frame metadata. Then, edge contour recognition is performed on this local image. The region where the boundary lines are located is determined by calculating the gradient change of pixel gray values. A sliding window is used to scan the image region and judge the gray value changes between adjacent pixels in the horizontal and vertical directions. When the change value exceeds the set edge recognition threshold, the pixel is marked as an edge point. This threshold is dynamically adjusted according to the overall brightness distribution of the image. If the average brightness of the image is below 100, the threshold is set to 15; if the brightness is between 100 and 180, it is set to 20; and if it is above 180, it is set to 25. After completing the initial edge point extraction, facial organ regions are segmented on the image based on the regions defined by facial structural features. The image is divided into three parts: the upper 1 / 5 for the eyebrow region, the middle 2 / 5 for the eye and nose region, and the lower 2 / 5 for the mouth and jaw region. Within each region, corresponding boundary lines are extracted. During this process, point coordinates are extracted for each line, and the two-dimensional coordinates of all edge points on each line are recorded as a trajectory point set. Then, a continuity analysis is performed on the trajectory point set to determine if the pixel distance between any two adjacent points is greater than twice the average distance between adjacent points. If so, the point is identified as a breakpoint and removed from the trajectory set. For example, if the average distance between consecutive points in an eyebrow boundary trajectory is 3 pixels, and an anomaly of 7 pixels appears in a segment, then that point in that segment is identified as a breakpoint and deleted. This extraction and removal process is repeated until the facial regions in all frames are extracted. Finally, multiple trajectory point sets consisting of the eyebrow, eye, nose, and mouth regions are constructed, forming a complete set of region boundary trajectories.

[0029] The trajectory continuity submodule calls the coordinates of trajectory points in the trajectory set at the region boundary, calculates the coordinate difference of each point between adjacent frames, compares it with the jump logic reference value, determines that points greater than the jump logic reference value are abnormal jump points and removes them, and aggregates the remaining coordinate points according to the frame order to generate a continuous trajectory point sequence. First, pair the trajectory points of the same facial region in adjacent frames according to frame number, and calculate the coordinate change difference of each pair of trajectory points between adjacent frames. The difference is calculated by calculating the difference between the horizontal and vertical coordinates separately, and then summing the results in both directions to determine the magnitude of the change. This determines whether the current point has a jump. If the jump exceeds the jump threshold, the point is considered an abnormal jump point and is removed. The jump threshold is set based on the movement amplitude of the facial region: 8 pixels for the eyebrow region, 10 pixels for the eye region, 6 pixels for the nose region, and 12 pixels for the corner of the mouth region. For example, when processing the trajectory of the left eye region... If a trajectory point in the current frame changes by 6 pixels horizontally and 7 pixels vertically compared to the previous frame, the merged change is 13 pixels, exceeding the baseline value of 10 pixels. Therefore, this point is marked as a jump point and removed from the trajectory of this frame. This operation is performed cyclically across all frames, traversing all region trajectory point sets. After removing all abnormal jump points, the remaining trajectory points are re-aggregated according to the frame number to ensure that the trajectory points of each facial region form a temporally continuous recording sequence in all frames. Finally, they are arranged in chronological order to form a continuous trajectory point sequence for each region in multiple image frames, which is used for subsequent trajectory connection and structure generation processing.

[0030] The regional trajectory construction submodule integrates the point sequences of the same region into continuous data packets according to time based on the continuous trajectory point sequence, performs trajectory segment connection processing, constructs the trajectory structure, and generates regional trajectory set fragments. First, for each facial region, such as the left eyebrow, right eye, nose wing, and corner of the mouth, the continuous trajectory points extracted in each frame are integrated into a data sequence packet in chronological order. The sequence packet records the trajectory point changes for that region over the entire time range. Next, trajectory segment connection processing is performed on the trajectory points in the sequence packet. During this process, the end of the trajectory point in an adjacent frame is compared with the beginning of the trajectory point in the next frame, and the magnitude of the coordinate change between them is judged. If the magnitude of the change is less than a preset connection threshold, it is determined to be a connectable segment, and trajectory splicing is performed. This threshold is set based on the region's stability. For the eye area... The region is within 10 pixels, and the nasal alar region is within 7 pixels. If the endpoint coordinates of frame_020 and the starting coordinates of frame_021 in the eye region differ by only 5 pixels, they are directly connected to form a complete trajectory segment. This process is repeated throughout the entire sequence packet, splicing together all trajectory segments that meet the connection conditions. Segments that do not meet the conditions are kept independent. Finally, all trajectory segments are classified and stored in the trajectory set of the region, forming multiple continuous segments. Each segment records the stable motion trajectory of the corresponding facial region over a period of time, constructing a region trajectory set segment.

[0031] Specifically, such as Figure 2 , 5 As shown, the variable interval marking module includes: The frame sequence numbering submodule is based on the region trajectory set fragments. It reads the position index of the trajectory line segment in the image frame sequence, records the occurrence number of the point in each trajectory in the frame sequence, constructs the inter-frame continuous structure according to the numbering order, extracts the start and end frame numbers of the trajectory segment, and generates the trajectory frame sequence number set. First, the system reads the trajectory point information contained in each trajectory segment in the set, and then extracts the image frame number corresponding to each point in each trajectory. This number has been bound to the image frames one by one in the previous frame sequence generation module. During the extraction process, the system records the specific index position of each point in the frame sequence. The recording method is to form a number sequence by arranging the frame numbers corresponding to all points in the trajectory segment in chronological order. For example, if the points in a certain eye trajectory segment appear in frame_004, frame_005, frame_006, and frame_008 in sequence, then the trajectory number sequence is {4, 5, 6, 8}. Then, based on this number sequence... The start and end frame numbers are used to extract the start and end range of each trajectory. For the above trajectory, the start frame number is 4 and the end frame number is 8. Then, the continuity between adjacent frames in the number sequence is judged, and the number difference between consecutive frames is calculated. If the number difference between adjacent frames is 1, it is considered a consecutive frame segment. If the difference is greater than 1, it is considered that there is a missing frame in the middle. The missing segment is recorded and marked in the final trajectory structure. This process is repeated to extract the number, sort the number, judge the continuity and extract the start and end frames for all trajectory segments. Finally, the number set of each trajectory line segment in the frame sequence is constructed and the trajectory frame sequence number set is generated.

[0032] The interval filtering submodule calls the start and end frame numbers in the trajectory frame sequence number set, calculates the time interval sequence between the first and last frames for each trajectory, judges the trend of time interval change, filters the trajectory combination with continuous time interval and coordinate change exceeding the preset threshold, and generates a continuously changing trajectory group. The time interval is obtained by multiplying the frame number difference by the frame interval based on the frame rate. For example, if the system frame rate is 25 frames per second and the interval between each frame is 40 milliseconds, then a trajectory number from frame_010 to frame_020, spanning 10 frames, corresponds to a time interval of 400 milliseconds. Next, the time interval values ​​of all trajectories are compared to establish a time interval sequence. Each item in the sequence represents the time value corresponding to the start and end frames of a trajectory. After establishing the sequence, the system judges the changing trend between adjacent interval values ​​in the time interval sequence, using the difference in time intervals between adjacent trajectories as the judgment criterion. The value is determined to be stable and within a specified difference range, which is preset to be between 80 milliseconds and 200 milliseconds. If the time interval between two trajectories is 360 milliseconds and 440 milliseconds respectively, then the difference between the two is 80 milliseconds, which meets the set range. This is considered as the coordinate change exceeding the preset threshold and existing continuously. Trajectory combinations that meet the filtering conditions will be retained for the next step. Conversely, if the difference is less than 80 milliseconds or greater than 200 milliseconds, it is considered as the interval change being too small or too large, which does not meet the filtering criteria and will be eliminated. Finally, all trajectory combinations that meet the conditions of continuity and interval difference are filtered out and form a group of continuously changing trajectory groups.

[0033] The interval identifier generation submodule is based on the continuously changing trajectory group. It extracts the number information of the trajectory in the combination, classifies all the numbers and establishes a number mapping relationship table. The mapped numbers are divided into combinations and encoded into the index list to generate a trajectory interval combination identifier set. First, the system extracts the numbering information for each trajectory within each group. Each trajectory already has a unique number. For example, trajectory group A consists of trajectory numbers T01, T02, and T03. After extracting these numbers, the system categorizes them and assigns all trajectory numbers to their respective trajectory groups. Then, a mapping table is established for these numbers. The mapping table structure is defined as "Track Number → Group Number". The group number is generated auto-incrementing according to the order of the trajectory groups. For example, if group A is labeled as G01 and group B is labeled as G02, then the mapping relationship for trajectories T01, T02, and T03 is T01→G01, T02→G01, and T03→G01. This processing method ensures that all trajectory numbers can be quickly found in the structured data to determine their group affiliation. After the table is established, the system divides all trajectory numbers according to their group affiliation and encodes the trajectory numbers under the same group into an index list. The index list for each group lists the corresponding trajectory numbers in the format G01:[T01, T02, T03]. Finally, the index lists for all groups are summarized and output to generate a trajectory interval combination identifier set.

[0034] Specifically, such as Figure 2 , 6 As shown, the facial response focusing module includes: The trajectory classification submodule extracts the image position index corresponding to each trajectory number based on the trajectory interval combination identifier set, classifies the trajectories according to the facial structure coordinate system, divides all numbers into four categories: nose wing, eyebrow, corner of mouth, and eye, accumulates the number of times the trajectory appears in the image frame sequence, and generates a regional trajectory classification statistics table. First, the trajectory numbers contained in each combination are extracted, and all positions appearing in the image frames corresponding to the numbers are searched. The search method is to traverse the coordinate set recorded for the trajectory number in all image frames and read the coordinate position of the center point of the trajectory in each frame. The position index value is stored in the trajectory structure in the form of pixels. Then, all center point coordinates are substituted into a preset facial structure standard coordinate system to determine the region to which the trajectory belongs. This coordinate system is uniformly divided into three layers according to the image resolution: upper, middle, and lower. 0% to 25% of the vertical height of the image is defined as the eyebrow region, 25% to 65% as the eye region, and 65% to 100% as the corner of the mouth region. The system classifies the trajectory points according to the density concentration of the coordinate distribution. If more than 80% of the coordinates of a trajectory are located within the eye region, it is classified as an eye trajectory. If the points are mainly concentrated in the eyebrow or corner of the mouth region, it is classified accordingly. After classification, the system counts the number of times each trajectory appears in the image frame sequence. It iterates through the set of frame numbers in which the trajectory number appears and counts the number of duplicates. For example, trajectory T005 appears in frame_008 to frame_012, a total of 5 frames, and is counted as 5 times. All trajectory numbers are sorted into four categories according to their classification. Each category records the number, region, and number of times each trajectory appears. Finally, the data is summarized to form a regional trajectory classification statistics table for subsequent interlacing detection processing.

[0035] The interlacing determination submodule calls the trajectory number and frame index information in the regional trajectory classification statistics table, retrieves the position number of any two types of trajectories in adjacent frames, performs range matching operation on the number interval, determines whether there is number overlap, and generates a list of trajectory interlacing numbers. First, all trajectories in the statistical table are categorized and read according to region type. For example, all trajectory numbers for eyebrow and mouth corner categories are extracted. Then, the set of frame numbers corresponding to each trajectory in the image frame sequence is read. Next, each pair of trajectories between any two categories is paired and combined. The frame number sequences of the two trajectory numbers are extracted from the combination, and an inter-frame overlap detection operation is performed. This involves comparing whether the two trajectories share a common frame number in their respective frame number sets. If at least one frame number is the same, the next step, range matching, is performed. In range matching, the boundary coordinate indices of the two trajectories under that frame number are read, including the pixel positions of the upper left and lower right corners. The length of the overlapping segment in the horizontal and vertical coordinates of the two trajectory regions is calculated and compared with the minimum boundary width and height of the two trajectories. When the horizontal overlap reaches 3 times the minimum width... If the overlap is greater than 0% and the vertical overlap reaches more than 20% of the minimum height, then the two trajectories are judged to have effective overlap in the frame. For example, trajectories T005 and T011 are located in regions (x=100−140,y=60−90) and (x=120−160,y=80−100) respectively in frame_015. The horizontal overlap is 20 pixels, accounting for 50% of the 40-pixel width of T005, and the vertical overlap is 10 pixels, accounting for 50% of the minimum height of 20 pixels. This meets the set judgment criteria, so frame_015 is marked as a frame with two intersecting trajectories. The intersecting information is recorded together with the trajectory number pair and the frame number. The trajectory intersecting number list is updated every time a pair is completed. Finally, all trajectory number pairs that meet the intersecting conditions and their corresponding frame numbers are collected and organized to form the trajectory intersecting number list.

[0036] The overlapping relationship submodule is based on the trajectory interlacing number list, filters the number combinations with continuous interlacing relationships, encodes and organizes the trajectory sequences that meet the set continuous conditions, establishes a classification and combination sequence structure, sorts the interlacing sequences according to the starting frame index and writes them into the mapping structure set, and generates the region arrangement overlapping relationship sequence. First, all track number pairs in the list are sorted in ascending order by frame number. Then, for each track number pair, it is determined whether there is a continuous overlapping relationship in the sorted frame numbers. The criterion is that the track pair must appear consecutively in at least three adjacent frame numbers. For example, if track pairs T005 and T011 are recorded in frames 15, 16, and 17, the pair is considered to meet the continuous overlapping condition. If they only appear in frames 15 and 17, and are missing in frame 16, the continuity criterion is not met, and the pair is removed. After filtering, the system selects all the remaining consecutively overlapping tracks. The track number pairs are re-encoded and assigned unique combination identifiers. Each combination identifier corresponds to a number pair and a frame interval range. For example, the number pair T005–T011 is assigned a combination ID of C01, a starting frame number of 15, and an ending frame number of 17. This combination information is organized into structural record entries, which include fields such as combination ID, track number pair, and frame start and end numbers. Then, all structural entries are reordered according to their starting frame numbers and written into the mapping structure set. Each record in the mapping structure set can directly reflect the specific combination relationship and occurrence position of the intersecting tracks, ultimately forming a sequence of overlapping relationships in the region arrangement.

[0037] Specifically, such as Figure 2 , 7 As shown, the state expression output module includes: The tag matching submodule extracts trajectory identification information from the number combinations based on the overlapping relationship sequence of the region arrangement, calls the expression tag mapping directory registered in the smart glasses, matches the number combinations with the corresponding tag numbers in the directory, performs index comparison for each number combination, records the tag number results, and generates a tag number correspondence table. First, the system reads the trajectory number combination information from the sequence one by one, extracting the individual trajectory number contained in each combination as the identification basis. The system groups two trajectory numbers in the number combination and performs standardized sorting to ensure that the combination order is not affected by the original arrangement during matching. For example, if the trajectory combination is {T011, T005}, it is converted to {T005, T011} for subsequent matching. Next, the system calls the expression tag mapping directory pre-stored in the smart glasses device. This directory consists of multiple key-value items, where the key is the trajectory number combination and the value is the corresponding emotion tag number. Each group of numbers has been associated with a specific expression state through the facial motion acquisition process. For example, {T005, T011} is mapped to tag number L03. The system will... The standardized number combinations are compared one by one with the key items in the directory. The comparison logic is a complete match principle, that is, the trajectory number in the combination is considered to be a successful match only if it is completely consistent with the number in the directory key. After a successful match, the corresponding tag number of the group is recorded. If the current combination is {T007, T014} and there is a corresponding item L05 in the directory, the system binds the combination with the tag number L05 and writes it into the tag number correspondence table. If there is no corresponding item in the directory, the tag number result of the combination is a null placeholder and does not participate in the subsequent encoding and processing. Only the matching failure record is retained for tracking. After all combinations have completed the matching process, the system outputs the tag number and its corresponding trajectory number combination in a structured manner to form a tag number correspondence table for subsequent tag encapsulation steps.

[0038] The tag encapsulation submodule calls the tag number results recorded in the tag number correspondence table, removes duplicate numbers and extracts all differentiated tag numbers, integrates the tag content associated with the tag number in turn, merges and encapsulates all content, and generates a tag encapsulation content set. First, all tag numbers in the table are read uniformly to generate a list of number sequences. This sequence may contain duplicate numbers and null values. The system first performs a deduplication operation. The deduplication process involves judging each number item by item and adding the unique number to the result set. Duplicate numbers are skipped, and null numbers are directly excluded and not included in the set of valid tag numbers. For example, if the original number sequence is {L03, L05, L03, L02, L05, null}, then the set of valid numbers after deduplication will be {L02, L03, L05}. Then, the system calls the corresponding tag text content in the emoji tag mapping directory in the order of each number in the set of valid numbers. Each tag number points to a short emotion description. For example, L03 corresponds to "eyes widening with eyebrows raised", L05 corresponds to "mouth slightly open and stretched to the left and right", and L02 corresponds to "increased blinking frequency". The system reads these tag contents and integrates them into a unified text unit. The integration process is arranged in numerical order, and the connection method uses semicolons to separate paragraph content. All the text is concatenated into "increased blinking frequency; eyes widening with eyebrows raised; mouth slightly open and stretched to the left and right". The integrated content is a tag-encapsulated content unit. The system adds it to the tag-encapsulated content set. Each encapsulated unit in this set is composed of a set of unique tag number contents. The content set finally summarizes all the differentiated tag contents that have appeared, and maintains its stable order and structural integrity.

[0039] The emotion output submodule encapsulates the content set according to the tags, imports all the encapsulated tag content into the result output channel, aggregates and encapsulates the content structure and numbers it before outputting it to the recognition structure to generate the emotion recognition result of the injured person. First, the system reads all the encapsulated tag texts in the content set one by one and imports them into the result output channel. This channel is preset as a temporary storage structure for emotion recognition results. The system numbers each encapsulated content in sequence, starting from R001 and incrementing sequentially. For example, the first encapsulated content is "increased blinking frequency; wide eyes accompanied by raised eyebrows; slightly open mouth and stretching left and right", so its number is marked as R001, the second is R002, and so on. After numbering, the system merges each number with the corresponding encapsulated content into a structured output unit. Each output unit consists of a number field and an emotion description field, and is uniformly written into the output structure cache list. Then, the system aggregates this structure list. The aggregation process does not change the original content order, but only organizes the structural hierarchy to form a complete output structure with index number, encapsulated text, and record position. Finally, the system outputs this structure to the emotion recognition result module, which writes the output structure into the recognition result record table, which can be used for subsequent querying, archiving, or visualization operations, thus completing the generation of the patient's emotion recognition results.

[0040] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A smart glasses system for recognizing the emotions of injured persons in complex emergency situations, characterized in that the system... include: The facial data capture module collects images and thermal information acquired by the smart glasses' camera and temperature sensing area during operation, reads the image frame number and thermal timestamp, removes data without number and data with discontinuous time, and generates frame-encoded image sequence groups in chronological order. The region trajectory extraction module, based on the frame-encoded image sequence group, marks the facial region, extracts the boundary trajectories of eyebrows, eyes, nose wings, and corners of the mouth, calculates the coordinate difference between the trajectory points in the preceding and following frames, retains continuous trajectory points, and generates a set of region trajectory fragments. The variable interval marking module extracts the region trajectory set fragments, records the frame index and number, constructs the inter-frame change structure, identifies trajectory combinations that are continuous in time and whose coordinate changes exceed a preset threshold, and generates a trajectory interval combination identifier set. The facial response focusing module classifies trajectories according to facial position based on the trajectory interval combination identifier set, counts the frequency, judges the number overlap, retains continuous interlaced combinations, and generates a sequence of overlapping relationship of regional arrangement; The state expression output module extracts the number combination of the overlapping relationship sequence of the regions, maps it to the facial region label, encapsulates the content of the label, and inputs it into the output channel to obtain the injury patient's emotion recognition result.

2. The intelligent glasses system for recognizing the emotions of injured persons in complex emergency situations according to claim 1, characterized in that: The frame-encoded image sequence group includes image frame sequence number, corresponding thermal timestamp, and mapping relationship between image and thermal information. The region trajectory set fragment includes facial region boundary line, feature part trajectory outline, and continuous inter-frame coordinate sequence. The trajectory interval combination identifier set includes trajectory start and end frame number, inter-trajectory time interval, and trajectory number mapping list. The region arrangement overlap relationship sequence includes trajectory category grouping, number of intra-frame trajectories, and overlap number combination. The injured person emotion recognition result includes number combination matching record, expression label name, and label encapsulation output content.

3. The intelligent glasses system for recognizing the emotions of injured persons in complex emergency situations according to claim 1, characterized in that: The combination of temporally continuous and differentiated trajectories refers to a set of facial region trajectories that have coordinate differences in adjacent frames but are temporally continuous.

4. The intelligent glasses system for recognizing the emotions of injured persons in complex emergency situations according to claim 1, characterized in that: The continuous interleaved combination refers to a combination of regions whose numbers appear alternately in time and whose trajectories are less than a set continuity threshold in space.

5. The intelligent glasses system for recognizing the emotions of injured persons in complex emergency situations according to claim 1, characterized in that, The facial data capture module includes: The synchronous calibration submodule is based on the image data set and thermal information record set captured by the camera area and temperature sensor embedded in the structure of the smart glasses during the working period. It obtains the image frame sequence number and thermal information timestamp content, determines whether the number and timestamp are missing or misaligned, removes the corresponding abnormal data frames and time records, and generates a synchronous index removal sequence. The image-thermal mapping submodule calls the synchronous index elimination sequence to filter and arrange the image data groups and thermal information in chronological order, establishes a key-value correspondence between each group of image frames and thermal parameters, and generates an image-thermal information mapping set. The frame sequence generation submodule extracts the image frame number and re-encodes it according to the time sequence structure in the image heat map set. The corresponding frame image data are arranged in order to generate a frame-encoded image sequence group.

6. The intelligent glasses system for recognizing the emotions of injured persons in complex emergency situations according to claim 1, characterized in that, The region trajectory extraction module includes: The boundary trajectory extraction submodule, based on the frame-encoded image sequence group, obtains the sub-image content marked as the facial region in each frame, identifies edge contour lines, separates the boundary lines of eyebrows, eyes, nose wings, and corners of the mouth, extracts the trajectory point coordinates of the region and removes breakpoints, and generates a set of region boundary trajectories. The trajectory continuity submodule calls the coordinates of trajectory points in the trajectory set of the region boundary, calculates the coordinate difference of each point between adjacent frames, compares it with the set jump logic benchmark value, determines that the jump point is abnormal and removes it, and aggregates the remaining coordinate points according to the frame order to generate a continuous trajectory point sequence. The regional trajectory construction submodule integrates the point sequences of the same region into continuous data packets according to time based on the continuous trajectory point sequence, performs trajectory segment connection processing, constructs the trajectory structure, and generates regional trajectory set fragments.

7. The intelligent glasses system for recognizing the emotions of injured persons in complex emergency situations according to claim 1, characterized in that, The variable interval marking module includes: The frame sequence numbering submodule reads the position index of the trajectory line segment in the image frame sequence based on the region trajectory set fragment, records the occurrence number of the point in each trajectory in the frame sequence, constructs the inter-frame continuous structure according to the numbering order, extracts the start and end frame numbers of the trajectory segment, and generates the trajectory frame sequence number set. The interval filtering submodule calls the start and end frame numbers in the trajectory frame sequence number set, calculates the time interval sequence between the first and last frames for each trajectory, judges the trend of time interval change, filters the trajectory combination with continuous time interval and coordinate change exceeding the preset threshold, and generates a continuously changing trajectory group. The interval identifier generation submodule extracts the number information of the trajectories in the combination based on the continuously changing trajectory group, classifies all the numbers and establishes a number mapping relationship table, encodes the mapped numbers into the index list after dividing them into combinations, and generates a trajectory interval combination identifier set.

8. The intelligent glasses system for recognizing the emotions of injured persons in complex emergency situations according to claim 1, characterized in that, The facial response focusing module includes: The trajectory classification submodule extracts the image position index corresponding to each trajectory number based on the trajectory interval combination identifier set, classifies the trajectories according to the facial structure coordinate system, divides all numbers into four categories: nose wing, eyebrow, corner of mouth, and eye, accumulates the number of times the trajectory appears in the image frame sequence, and generates a regional trajectory classification statistics table. The interlacing determination submodule calls the trajectory number and frame index information in the regional trajectory classification statistics table, retrieves the position number of any two types of trajectories in adjacent frames, performs a range matching operation on the number interval, determines whether there is number overlap, and generates a list of trajectory interlacing numbers. The overlapping relationship submodule, based on the trajectory interlacing number list, filters out number combinations with continuous interlacing relationships, encodes and organizes trajectory sequences that meet the set continuous conditions, establishes a classification and combination sequence structure, sorts the interlacing sequences according to the starting frame index and writes them into the mapping structure set, and generates a region arrangement overlapping relationship sequence.

9. The intelligent glasses system for recognizing the emotions of injured persons in complex emergency situations according to claim 1, characterized in that, The state expression output module includes: The tag matching submodule extracts trajectory identification information from the numbered combinations based on the overlapping relationship sequence of the regions, calls the expression tag mapping directory registered in the smart glasses, matches the numbered combinations with the corresponding tag numbers in the directory, performs index comparison for each numbered combination, records the tag number results, and generates a tag number correspondence table. The tag encapsulation submodule calls the tag number results recorded in the tag number correspondence table, performs duplicate number deduplication and extracts all differentiated tag numbers, integrates the tag content associated with the tag number in turn, merges and encapsulates all content, and generates a tag encapsulation content set. The emotion output submodule encapsulates the content set based on the tags, imports all the encapsulated tag content into the result output channel, aggregates and encapsulates the content structure and labels it with numbers, and then outputs it to the recognition structure to generate the emotion recognition result of the injured person.

10. The intelligent glasses system for recognizing the emotions of injured persons in complex emergency situations according to claim 9, characterized in that, The process of deduplicating duplicate numbers and extracting all differentiated label numbers refers to extracting the set of all numbers that have uniqueness and distinguishability among the label numbers, after excluding duplicate numbers.

Images