Intention recognition method and device

Abstract

This application provides an intent recognition method and apparatus, belonging to the field of intent recognition technology. The method includes: acquiring capacitance information sent by a near-field sensing module and eye-tracking information sent by an eye-tracking recognition module; the capacitance information includes capacitance change regions and corresponding capacitance change information; the eye-tracking information includes eye-tracking trajectory regions and corresponding dwell times; determining a first intent region for the operator based on the capacitance change regions and capacitance change information; determining a second intent region for the operator based on the eye-tracking trajectory regions and dwell times; determining a target intent region for the operator based on the first and second intent regions; recognizing the operator's intent based on the target intent region to obtain the target intent, and displaying a pop-up reminder on a touchscreen based on preset standard process information and the target intent. This application can, to a certain extent, replace real-time instruction from instructors, improving the relevance and effectiveness of training.

Description

Technical Field

[0001] This application belongs to the field of intent recognition technology, and more specifically, relates to an intent recognition method and device applied to an elevator teaching system. Background Technology

[0002] Currently, elevators play a vital role in people's daily lives and work as a means of transportation. To ensure the safe operation of elevators and the safety of elevator industry personnel, all elevator workers must undergo professional learning and training, and pass an examination before they can engage in elevator-related work.

[0003] Currently, elevator operator training largely falls into two categories: one relies on real elevators and shafts, and the other on virtual simulation teaching systems. Virtual teaching systems are more widely used due to their lower risk. However, long-term training and surveys have revealed that inexperienced trainees require instructor guidance or positive feedback before each step to build confidence. Instructors, with limited time and energy, cannot provide real-time, advance guidance and confirmation of intent for each trainee. This uncertainty leads to errors, hindering the trainee's ability to quickly grasp elevator balance coefficient measurement knowledge and potentially damaging training equipment.

[0004] Therefore, an intent recognition method is needed to replace real-time instructor guidance to some extent, thereby improving the relevance and effectiveness of training. Summary of the Invention

[0005] The purpose of this application is to provide an intent recognition method and device to replace real-time guidance from instructors to a certain extent, thereby improving the relevance and effectiveness of training.

[0006] A first aspect of this application provides an intent recognition method applied to a controller in an elevator teaching simulation system. The elevator teaching simulation system further includes: a touchscreen, a near-field sensing module, and an eye-tracking recognition module. The near-field sensing module is used to sense capacitance changes within a preset range outside the touchscreen, and the eye-tracking recognition module is used to acquire changes in the operator's eye movements. The intent recognition method includes: Acquire capacitance information sent by the near-field sensing module and eye movement information sent by the eye movement recognition module; capacitance information includes capacitance change regions and corresponding capacitance change information; eye movement information includes eye movement trajectory regions and corresponding dwell times. The operator's primary intention area is determined based on the capacitance change region and capacitance change information. Determine the operator's second intention area based on eye-tracking trajectory area and dwell time; The operator's target intent region is determined based on the first intent region and the second intent region. Based on the target intent area, the operator's intent is identified to obtain the target intent. Based on the preset standard process information and the target intent, a pop-up reminder is displayed on the touch screen. The target intent is used to represent the operator's intended operation, and the pop-up reminder is used to guide the operator's subsequent operations.

[0007] A second aspect of this application provides an intent recognition device applied to a controller in an elevator teaching simulation system. The elevator teaching simulation system further includes: a touch screen, a near-field sensing module, and an eye-tracking recognition module. The near-field sensing module is used to sense capacitance changes within a preset range outside the touch screen, and the eye-tracking recognition module is used to acquire changes in the operator's eye movements. The intent recognition device includes: The data acquisition module is used to acquire capacitance information sent by the near-field perception module and eye movement information sent by the eye movement recognition module; the capacitance information includes capacitance change areas and capacitance change information corresponding to the capacitance change areas; the eye movement information includes eye movement trajectory areas and the dwell time corresponding to the eye movement trajectory areas. The first intent recognition module is used to determine the operator's first intent area based on the capacitance change area and capacitance change information. The second intent recognition module is used to determine the operator's second intent area based on the eye movement trajectory area and dwell time. The third intent recognition module is used to determine the operator's target intent region based on the first intent region and the second intent region. The intent recognition and pop-up module is used to recognize the operator's intent based on the target intent area, obtain the target intent, and display a pop-up reminder on the touch screen based on the preset standard process information and the target intent. The target intent is used to represent the operator's intended operation, and the pop-up reminder is used to guide the operator's subsequent operations.

[0008] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the intent recognition method described above.

[0009] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the intent recognition method described above.

[0010] The beneficial effects of the intent recognition method and apparatus provided in this application are as follows: This application embodiment collects capacitance information and eye-tracking information based on a near-field sensing module and an eye-tracking recognition module, respectively. This dual-module, multi-dimensional data acquisition avoids the recognition bias of a single sensing method. First and second intent regions are determined based on the two types of information and then fused to obtain the target intent region. This captures the operator's intention, enabling pre-operation intention prediction and recognition, replacing real-time confirmation of the operator's intention by the instructor, and solving the problem of limited instructor time for individual guidance. Simultaneously, based on the identified target intent, this application embodiment provides subsequent operation guidance to the operator through pop-up reminders on the touchscreen, combined with preset standard procedure information. This alleviates the uncertainty of operation for inexperienced operators, reduces operational errors caused by ambiguous intentions, avoids damage to training equipment, and allows operators to learn according to standard procedures, improving their mastery of elevator balance coefficient measurement knowledge. Therefore, this application embodiment achieves accurate intent recognition through dual-module data acquisition and intent region fusion, and completes operation guidance through pop-up reminders of standard procedures, to a certain extent replacing real-time instructor guidance, improving the relevance and effectiveness of elevator teaching simulation system training. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating an intent recognition method provided in an embodiment of this application; Figure 2 This is a structural block diagram of an intent recognition device provided in an embodiment of this application; Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0014] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0015] In one embodiment of this application, the intent recognition method is applied to the controller of an elevator teaching simulation system. The elevator teaching simulation system further includes a touch screen, a near-field sensing module, and an eye-tracking recognition module. The near-field sensing module is used to sense capacitance changes within a preset range outside the touch screen, and the eye-tracking recognition module is used to acquire changes in the operator's eye movements.

[0016] In this embodiment, the elevator teaching simulation system is a simulation teaching platform that provides training for elevator practitioners in core skills such as balance coefficient determination. It integrates functional modules such as mechanical simulation, electrical control, human-computer interaction, and data acquisition. This embodiment will not elaborate on the simulation system.

[0017] In this embodiment, the touchscreen serves as a visual interface for operators to trigger operation commands and display information. The near-field sensing module is a non-contact sensing module based on the principle of capacitive sensing, deployed on the outside of the touchscreen. It possesses the capability to detect and quantify changes in the capacitive field, and its sensing range is a pre-calibrated effective spatial area, typically within 5cm of the outside of the touchscreen. The eye-tracking recognition module is a visual sensing module based on eye-tracking technology (such as infrared corneal reflection). It can capture the dynamic process of the operator's eye movements and gaze focus shifts, achieving quantified acquisition and coordinate representation of gaze trajectory and dwell time. It serves as the acquisition end for visual intent data.

[0018] In this embodiment, capacitance change refers to the change in capacitance value generated when the operator's finger enters the effective acquisition field of the near-field sensing module, causing a change in the capacitance field distribution between the module's sensing electrodes and ground. This change is used to characterize the spatial position and movement state of the finger within a preset range. Eye movement change refers to the dynamic process of eye movements, such as the movement of the operator's gaze focus in space and the lingering of the gaze on the touchscreen display area. This reflects the operator's visual attention tendency and can be digitally represented through quantitative indicators such as gaze trajectory and lingering duration.

[0019] In this embodiment, the near-field sensing module essentially uses the principle of capacitive sensing to sense the position and distance of the operator's fingers. Therefore, infrared light can also be used for sensing. The detailed principle will not be elaborated here.

[0020] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an intent recognition method provided in an embodiment of this application. The method may include: S101-S105.

[0021] S101: Obtain capacitance information sent by the near-field sensing module and eye movement information sent by the eye movement recognition module.

[0022] In this embodiment, the capacitance information includes the capacitance change region and the corresponding capacitance change information within that region. The capacitance change region refers to the coordinated spatial area within the effective acquisition domain of the near-field sensing module where capacitance changes are detected, reflecting the spatial activity range of the finger within a preset range. This area can be transmitted to the controller in the form of a coordinate set. The capacitance change information refers to the quantified characteristic data of capacitance changes at each spatial point within the capacitance change region, including but not limited to capacitance strength and capacitance change timing. This data represents the degree, process, and trend of capacitance change.

[0023] In this embodiment, eye-tracking information includes eye-tracking trajectory regions and corresponding dwell times. The eye-tracking trajectory region refers to the coordinated spatial area covered by the trajectory formed by the operator's gaze focus moving in space, detected by the eye-tracking recognition module. Spatial matching with the touchscreen's operating area can be achieved through a coordinate mapping algorithm. Dwell time refers to the duration for which the operator's gaze focus remains within each area of ​​the eye-tracking trajectory region.

[0024] In this embodiment, the coordinate system refers to a two-dimensional coordinate system with the touch screen as the plane. The origin can be the lower left corner of the touch screen, the X-axis can be the lower edge of the touch screen with the rightward direction as positive, and the Y-axis can be the left edge of the touch screen with the upward direction as positive.

[0025] S102: Determine the operator's first intention area based on the capacitance change area and capacitance change information.

[0026] In this embodiment, the first intention area refers to the physical directional operation area that the operator's finger is about to point to or touch, determined from the effective operation area of ​​the touch screen by using capacitance information as data and algorithms such as spatial analysis, trajectory fitting, and threshold determination.

[0027] Specifically, the capacitance change information includes: capacitance strength and the timing of capacitance changes. The greater the capacitance strength, the smaller the distance between the finger and the touchscreen. The timing of capacitance changes refers to the time sequence in which the near-field sensing module detects changes in capacitance strength at various spatial points within the capacitance change area.

[0028] In one embodiment, determining the operator's first intention area based on the capacitance change region and capacitance change information includes: The capacitance change area is divided into several capacitance detection sub-regions by rasterization. Based on the capacitance change time sequence, the capacitance detection sub-regions are sorted in order of increasing capacitance strength to obtain the movement path of the operator's finger within a preset range outside the touch screen. The movement path is fitted with a trend to obtain the extended trajectory curve of the finger movement. Based on the capacitance strength and capacitance change time sequence, the region in the extended trajectory curve where the capacitance strength reaches the preset touch capacitance strength threshold is determined, and this region is determined as the first intention region.

[0029] In this embodiment, gridding refers to the spatial discretization operation performed on the capacitance change region. It divides the continuous capacitance change region into several non-overlapping and fully covered regular spatial units at a preset fixed scale. The capacitance detection sub-region is the smallest independent spatial detection unit obtained after gridding the capacitance change region.

[0030] In this embodiment, mathematical fitting operations can be performed on the spatial coordinate data of the movement path. Based on the existing coordinate points of the movement path, the trajectory trend function of the finger movement is solved, enabling mathematical prediction of the subsequent movement trajectory of the finger that has not yet occurred. The fitting operation can be polynomial fitting or linear fitting. The extended trajectory curve is the predicted trajectory curve of finger movement obtained from the trend fitting operation, which is the extension of the movement path in the direction of the touch screen.

[0031] In this embodiment, when the operator moves their finger on the touchscreen, the distance between the finger and the sensing electrode gradually decreases, and the corresponding capacitance strength changes from weak to strong. Moreover, this change shows a clear temporal sequence as the finger moves (capacitance change sequence). The controller uses the capacitance change sequence as the time axis to spatially sort each capacitance detection sub-region in order of capacitance strength from weak to strong. Then, the center coordinates of each sorted sub-region are connected sequentially to form a finger movement path with directional continuity.

[0032] In this embodiment, the preset touch capacitance strength threshold refers to a pre-calibrated critical capacitance strength value corresponding to the moment when the operator's finger is about to touch the touchscreen operating surface. The controller can extract the capacitance strength values ​​of all points on the extended trajectory curve, filter out the spatial region in the extended trajectory curve where the capacitance strength reaches the preset touch capacitance strength threshold, and determine this region as the first intention region, which is the final pointing area of ​​the operator's finger physical movement.

[0033] In this embodiment, the capacitance values ​​of all points on the extended trajectory curve can be determined based on the capacitance and the timing of capacitance changes. For example, the change in capacitance per unit length can be determined based on the current capacitance and the timing of capacitance changes, and then the capacitance values ​​of all points on the extended trajectory curve can be obtained based on the length of the extended trajectory curve.

[0034] S103: Determine the operator's second intention area based on the eye-tracking trajectory area and dwell time.

[0035] In this embodiment, the second intention area refers to the visually directional operation area that the operator is continuously visually focused on, determined from the effective operation area of ​​the touch screen by algorithms such as area filtering, duration statistics, and coordinate mapping based on eye-tracking information.

[0036] In one embodiment, determining the operator's second intention region based on the eye-tracking trajectory region and dwell time includes: The eye movement trajectory region is divided into multiple eye movement detection sub-regions by rasterization. The dwell time in each eye movement detection sub-region is counted, and eye movement detection sub-regions with dwell time less than a preset dwell threshold are removed to obtain multiple effective eye movement detection sub-regions; Multiple effective eye-tracking detection sub-regions are sorted from longest to shortest dwell time, and a predetermined number of consecutive effective eye-tracking detection sub-regions are selected to form the gaze-focusing core region; The gaze is focused on the core area and mapped onto the effective operating coordinate system of the touchscreen to obtain a mapped area that matches the operating area of ​​the touchscreen. This mapped area is then designated as the second intent area. The preset dwell threshold can be set to 100ms, and the preset number can be set to 5.

[0037] In this embodiment, the gridding criteria and process for the eye movement trajectory region are the same as those for the capacitance change region. After gridding, multiple eye movement detection sub-regions are obtained. All eye movement detection sub-regions are traversed to quantify the dwell time of each sub-region. Using a preset dwell time threshold as the criterion, eye movement detection sub-regions with dwell times less than the threshold are removed. Dwell time data corresponding to invalid eye movements such as brief gaze shifts are filtered out, retaining only valid data reflecting the operator's active visual attention. This eliminates the interference of invalid data on intent determination, and the remaining sub-regions are the valid eye movement detection sub-regions. All valid eye movement detection sub-regions are sorted from longest to shortest dwell time, with the highest-ranked sub-regions selected first. Next, based on a preset quantity threshold, a preset number of consecutive sub-regions are selected (to avoid blurring of the attention area caused by the aggregation of discrete sub-regions), and these sub-regions are aggregated into the gaze focus core region. The aggregated core region has the visual attention priority with the longest dwell time and spatial continuity. Finally, the original coordinates (eye-tracking module coordinate system) of the core area of ​​gaze focus are mapped to the effective operation coordinate system of the touch screen. During the mapping process, the origin, coordinate axis direction and scale ratio of the two coordinate systems are calibrated to ensure that the mapped area coordinates are adapted to the effective operation area of ​​the touch screen, and the mapped area is determined as the second intent area.

[0038] S104: Determine the operator's target intent region based on the first intent region and the second intent region.

[0039] In one embodiment, determining the operator's target intent region based on the first intent region and the second intent region includes: determining the overlapping area of ​​the first intent region and the second intent region as the operator's target intent region.

[0040] In this embodiment, the target intent area refers to the intersection of the spatial coordinates of the first intent area and the second intent area in the effective operation coordinate system of the touch screen, and is the effective operation area of ​​the touch screen that has both physical limb orientation and visual attention.

[0041] The first intention area only reflects the physical pointing tendency of the limbs, and the second intention area only reflects the visual attention tendency. Both single-dimensional intention areas are prone to misjudgment, such as when a finger accidentally points to an area but the gaze does not focus on it, or when the gaze focuses on an area but the finger does not point to it. Therefore, by filtering overlapping areas through spatial intersection operations, dual verification at the physical and visual levels can be achieved, improving the accuracy of target intention area determination.

[0042] S105: Based on the target intent area, the operator's intent is identified to obtain the target intent, and a pop-up reminder is displayed on the touch screen based on the preset standard process information and the target intent.

[0043] In this embodiment, the target intent is used to characterize the operator's intended operation, and the pop-up reminder is used to guide the operator's subsequent operations.

[0044] In this embodiment, the target intent refers to the operable instruction area and its corresponding elevator teaching operation instructions that match the operator's actual operating intent. The preset standard process information refers to the standardized operational process structured data corresponding to the elevator teaching simulation system, pre-stored in the controller. This includes the operation steps, operation sequence, compliant operation instruction set, and process stage judgment criteria for teaching content such as elevator balance coefficient measurement and adjustment. The pop-up reminder is a visual guidance information generated and displayed next to the target intent area on the touchscreen based on the compliance matching result between the target intent and the preset standard process information. It provides feedback and guidance.

[0045] In one embodiment, a pop-up reminder is displayed on the touchscreen based on preset standard process information and target intent, including: Standard operations are determined based on preset standard process information; If the operation command corresponding to the target intent conforms to the standard operation, a confirmation pop-up window will appear next to the target intent area on the touch screen; the confirmation pop-up window is used to provide positive feedback to the operator's operation intent. If the operation command corresponding to the target intent does not conform to the standard operation, a guidance pop-up window will appear next to the target intent area on the touch screen; the guidance pop-up window is used to guide the operator to perform the standard operation.

[0046] In this embodiment, standard operation refers to the only compliant operation instruction that is allowed to be executed at a given stage, determined based on preset standard process information and the current stage of the elevator teaching simulation system. The operation instruction refers to the specific execution instruction corresponding to the target intent in the elevator teaching simulation system, such as a traction machine start instruction, a traction machine braking instruction, or an elevator upward speed setting instruction.

[0047] In this embodiment, confirmation pop-up reminders are used to provide feedback to operators that their operational intentions conform to the standard procedure, and are considered positive interactive outputs. Guidance pop-up reminders are used to correct deviations in the operator's intentions, guiding them back to the standard teaching procedure.

[0048] As can be seen from the above, the embodiments of this application collect capacitance information and eye movement information based on the near-field perception module and the eye-tracking recognition module, respectively. Through multi-dimensional data acquisition using dual modules, the recognition bias of a single perception method is avoided. Then, based on the two types of information, the first and second intent regions are determined and fused to obtain the target intent region. This can capture the operator's operational intent, achieving pre-operation intent prediction and recognition, replacing the instructor's real-time confirmation of the operator's intent, and solving the problem of instructors having limited energy and being unable to provide individual guidance. Simultaneously, based on the identified target intent, the embodiments of this application provide subsequent operation guidance to the operator through pop-up reminders on the touchscreen, combined with preset standard process information. This alleviates the operational uncertainty of inexperienced operators, reduces operational errors caused by ambiguous intent, avoids damage to training equipment, and allows operators to learn according to the standard process, improving their efficiency in mastering elevator balance coefficient measurement knowledge. Therefore, the embodiments of this application achieve accurate intent recognition through dual-module data acquisition and intent region fusion, and complete operation guidance through pop-up reminders of the standard process, to a certain extent replacing real-time instructor guidance, improving the pertinence and effectiveness of elevator teaching simulation system training.

[0049] In one embodiment of this application, the operator's intent is identified based on the target intent region to obtain the target intent, including: If the target intent area contains only one operable instruction area, then the operable instruction area is determined as the target intent, so as to realize the recognition of the operator's intent; In response to the presence of multiple operable instruction regions within the target intent region, the eye-tracking features corresponding to each operable instruction region are determined, and the capacitance features corresponding to each operable instruction region are determined based on the extended trajectory curve. Based on the eye-tracking and capacitance features corresponding to multiple operable instruction regions, one operable instruction region is determined from the multiple operable instruction regions as the target intent to achieve operator intent recognition. Among these features, the eye-tracking features are used to characterize the operator's visual attention to each operable instruction region, and the capacitance features are used to characterize the operator's finger movement trajectory pointing towards each operable instruction region.

[0050] In this embodiment, the operable instruction area refers to an independent functional space area that is pre-calibrated under the effective operating coordinate system of the touch screen and bound to a unique elevator teaching operation instruction. There are clear coordinate boundaries and functional mapping relationships on the touch screen. For example, instructions such as "traction machine start" and "balance coefficient calibration" all correspond to a dedicated operable instruction area.

[0051] In this embodiment, if the target intent region contains only one operable command region, no additional feature analysis is required, and the operable command region can be directly identified as the target intent. When the target intent region contains multiple operable command regions, two-dimensional feature extraction should be performed for each operable command region, and each operable command region should be analyzed based on the extracted eye-tracking and capacitance features. In this embodiment, eye-tracking features include dwell time and the number of times the object is gazed upon; capacitance features include capacitance intensity and capacitance change rate.

[0052] In one implementation, determining an operable instruction region as the target intent from multiple operable instruction regions based on eye-tracking and capacitive features corresponding to those regions includes: The dwell time, number of gazes, capacitance intensity, and capacitance change rate of each operable instruction region are normalized. For each operable instruction region, the intent score corresponding to that operable instruction region is determined based on the normalized dwell time, normalized number of gazes, normalized capacitance intensity, and normalized capacitance change rate. The operable instruction region with the highest intent score among all operable instruction regions is taken as the target intent.

[0053] In this embodiment, the number of times someone is looked at refers to the cumulative number of times an operator's gaze enters and lingers in a single operable command area for a duration ≥ a preset lingering threshold. The capacitance change rate refers to the unit-time increment of capacitance intensity within a single operable command area as capacitance changes over time.

[0054] In this embodiment, dwell time, number of gazes, capacitance intensity, and capacitance change rate can be normalized and mapped to the [0,1] interval. Since dwell time, number of gazes, capacitance intensity, and capacitance change rate are all positively correlated with the intent score, the normalized dwell time, normalized number of gazes, normalized capacitance intensity, and normalized capacitance change rate can be directly weighted to obtain the intent score. The weights for each weighted calculation can be set to 0.25. After calculating the intent scores for all operable instruction areas, a sorted list is generated from high to low scores; the operable instruction area at the top of the list (with the highest intent score) is selected as the target intent.

[0055] As can be seen from the above, this application embodiment uses a combination of capacitive and eye-tracking features to characterize the physical pointing tendency of the finger and the degree of visual attention, respectively. Compared with a single sensing method, this improves the accuracy and robustness of intent recognition, enabling the prediction of the operation intent before the student touches the screen, thus providing advance guidance. Secondly, for scenarios where there are multiple operable instruction areas within the target intent area, normalization processing and weighted intent scoring can accurately lock the unique target intent even under multi-button interference, avoiding ambiguity caused by overlapping areas. Thirdly, this application embodiment can automatically identify the student's operation intent and provide feedback based on standard procedures, to a certain extent replacing human instructors to achieve real-time, advance intent confirmation and operation guidance, alleviating the problem of limited instructor energy. Finally, through timely positive feedback and standardized guidance, this application embodiment can enhance the student's confidence in operation, reduce the probability of misoperation, protect the simulation equipment, and accelerate the student's mastery of skills such as elevator balance coefficient measurement, thereby improving training efficiency and teaching quality.

[0056] Corresponding to the intent recognition method in the above embodiments, Figure 2 This is a structural block diagram of an intent recognition device provided according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. Reference Figure 2 The intent recognition device 20 is applied to the controller of the elevator teaching simulation system. The elevator teaching simulation system also includes: a touch screen, a near-field sensing module, and an eye-tracking recognition module. The near-field sensing module is used to sense the capacitance change within a preset range outside the touch screen, and the eye-tracking recognition module is used to acquire the operator's eye movement changes. The intent recognition device 20 includes: a data acquisition module 21, a first intent recognition module 22, a second intent recognition module 23, a third intent recognition module 24, and an intent recognition and pop-up window module 25.

[0057] The data acquisition module 21 is used to acquire capacitance information sent by the near-field perception module and eye movement information sent by the eye movement recognition module; the capacitance information includes capacitance change areas and capacitance change information corresponding to the capacitance change areas; the eye movement information includes eye movement trajectory areas and dwell time corresponding to the eye movement trajectory areas. The first intent recognition module 22 is used to determine the operator's first intent area based on the capacitance change area and capacitance change information. The second intent recognition module 23 is used to determine the operator's second intent area based on the eye movement trajectory area and dwell time. The third intent recognition module 24 is used to determine the operator's target intent region based on the first intent region and the second intent region; The intent recognition and pop-up module 25 is used to recognize the operator's intent based on the target intent area, obtain the target intent, and display a pop-up reminder on the touch screen based on the preset standard process information and the target intent; wherein, the target intent is used to represent the operator's intended operation, and the pop-up reminder is used to guide the operator's subsequent operation.

[0058] In one embodiment of this application, the capacitance change information includes: capacitance strength and capacitance change timing; The first intent recognition module 22 is specifically used to perform grid-based division of the capacitance change region to obtain several capacitance detection sub-regions. Based on the capacitance change time sequence, the capacitance detection sub-regions are sorted in order of capacitance strength from weakest to strongest to obtain the movement path of the operator's finger within a preset range outside the touch screen. By performing trend fitting on the movement path, the extended trajectory curve of the finger movement is obtained; Based on the capacitance intensity and capacitance change time sequence, the region in the extended trajectory curve where the capacitance intensity reaches the preset touch capacitance intensity threshold is determined, and this region is determined as the first intention region.

[0059] In one embodiment of this application, the second intent recognition module 23 is specifically used to perform rasterization of the eye movement trajectory region to obtain multiple eye movement detection sub-regions; The dwell time in each eye movement detection sub-region is counted, and eye movement detection sub-regions with dwell time less than a preset dwell threshold are removed to obtain multiple effective eye movement detection sub-regions; Multiple effective eye-tracking detection sub-regions are sorted from longest to shortest dwell time, and a predetermined number of consecutive effective eye-tracking detection sub-regions are selected to form the gaze-focusing core region; By focusing the gaze on the core area and mapping it to the effective operating coordinate system of the touchscreen, a mapping area that matches the operating area of ​​the touchscreen is obtained, and this mapping area is determined as the second intent area.

[0060] In one embodiment of this application, the second intent recognition module 23 is specifically used to determine the operable instruction area as the target intent in response to the target intent area containing only one operable instruction area, so as to realize the intent recognition of the operator. In response to the presence of multiple operable instruction regions within the target intent region, the eye-tracking features corresponding to each operable instruction region are determined, and the capacitance features corresponding to each operable instruction region are determined based on the extended trajectory curve. Based on the eye-tracking and capacitance features corresponding to multiple operable instruction regions, one operable instruction region is determined from the multiple operable instruction regions as the target intent to achieve operator intent recognition. Among these features, the eye-tracking features are used to characterize the operator's visual attention to each operable instruction region, and the capacitance features are used to characterize the operator's finger movement trajectory pointing towards each operable instruction region.

[0061] In one embodiment of this application, eye movement features include dwell time and number of gazes; capacitance features include capacitance strength and capacitance change rate. The second intent recognition module 23 is further used to normalize the dwell time, number of gazes, capacitance intensity and capacitance change rate of each operable instruction area; For each operable instruction region, the intent score corresponding to that operable instruction region is determined based on the normalized dwell time, normalized number of gazes, normalized capacitance intensity, and normalized capacitance change rate. The operable instruction region with the highest intent score in each operable instruction region is taken as the target intent.

[0062] In one embodiment of this application, the intent recognition and pop-up module 25 is specifically used to determine standard operations based on preset standard process information; If the operation command corresponding to the target intent conforms to the standard operation, a confirmation pop-up window will appear next to the target intent area on the touch screen; the confirmation pop-up window is used to provide positive feedback to the operator's operation intent. If the operation command corresponding to the target intent does not conform to the standard operation, a guidance pop-up window will appear next to the target intent area on the touch screen; the guidance pop-up window is used to guide the operator to perform the standard operation.

[0063] In one embodiment of this application, the intent recognition and pop-up module 25 is further used to determine the overlapping area of ​​the first intent area and the second intent area as the target intent area of ​​the operator.

[0064] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the data acquisition module 21, the first intent recognition module 22, the second intent recognition module 23, the third intent recognition module 24, and the intent recognition and pop-up module 25 are shown.

[0065] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0066] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0067] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store device type information.

[0068] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation method described in the intent recognition method provided in the embodiments of this application, or they can execute the implementation method of the electronic device described in the embodiments of this application, which will not be repeated here.

[0069] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0070] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0071] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0072] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0073] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.

[0074] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0075] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0076] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0077] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An intent recognition method, characterized in that, A controller is used in an elevator teaching simulation system, which further includes a touchscreen, a near-field sensing module, and an eye-tracking recognition module. The near-field sensing module is used to sense capacitance changes within a preset range outside the touchscreen, and the eye-tracking recognition module is used to acquire changes in the operator's eye movements. The intention recognition method includes: Acquire capacitance information sent by the near-field sensing module and eye movement information sent by the eye movement recognition module; the capacitance information includes capacitance change regions and capacitance change information corresponding to the capacitance change regions; the eye movement information includes eye movement trajectory regions and dwell time corresponding to the eye movement trajectory regions; The operator's first intention area is determined based on the capacitance change region and the capacitance change information. The operator's second intention area is determined based on the eye-tracking trajectory area and the dwell time; The operator's target intent region is determined based on the first intent region and the second intent region; Based on the target intent area, the operator's intent is identified to obtain the target intent, and a pop-up reminder is displayed on the touchscreen based on preset standard process information and the target intent; wherein, the target intent is used to characterize the operator's intended operation, and the pop-up reminder is used to guide the operator's subsequent operations.

2. The intent recognition method as described in claim 1, characterized in that, The capacitance change information includes: capacitance strength and capacitance change timing; Determining the operator's first intention area based on the capacitance change region and the capacitance change information includes: The capacitance change region is divided into several capacitance detection sub-regions by gridding. Based on the capacitance change time sequence, the capacitance detection sub-regions are sorted in order of capacitance strength from weakest to strongest to obtain the movement path of the operator's finger within a preset range outside the touch screen. Trend fitting is performed on the movement path to obtain the extended trajectory curve of the finger movement; Based on the capacitance strength and the timing of capacitance changes, the region in the extended trajectory curve where the capacitance strength reaches a preset touch capacitance strength threshold is determined, and this region is defined as the first intention region.

3. The intent recognition method as described in claim 1, characterized in that, The determination of the operator's second intention region based on the eye-tracking trajectory region and the dwell time includes: The eye movement trajectory region is divided into multiple eye movement detection sub-regions by rasterization. The dwell time in each of the eye movement detection sub-regions is counted, and eye movement detection sub-regions with dwell time less than a preset dwell threshold are removed to obtain multiple effective eye movement detection sub-regions; The multiple effective eye movement detection sub-regions are sorted from longest to shortest dwell time, and a predetermined number of consecutive effective eye movement detection sub-regions are selected to form the gaze focusing core region; The core area of ​​the gaze focus is mapped to the effective operation coordinate system of the touch screen to obtain a mapped area that matches the operation area of ​​the touch screen, and this mapped area is determined as the second intention area.

4. The intent recognition method as described in claim 2, characterized in that, The step of identifying the operator's intent based on the target intent region to obtain the target intent includes: If the target intent area contains only one operable instruction area, then the operable instruction area is determined as the target intent, so as to realize the intent recognition of the operator; In response to the presence of multiple operable instruction regions within the target intent region, eye-tracking features corresponding to each operable instruction region are determined, and capacitance features corresponding to each operable instruction region are determined based on the extended trajectory curve. Based on the eye-tracking and capacitance features corresponding to the multiple operable instruction regions, one operable instruction region is determined from the multiple operable instruction regions as the target intent, thereby achieving intent recognition of the operator. The eye-tracking features characterize the operator's visual attention to each operable instruction region, and the capacitance features characterize the operator's finger movement trajectory's directional tendency towards each operable instruction region.

5. The intent recognition method as described in claim 4, characterized in that, The eye movement characteristics include dwell time and number of times the eye is gazed upon; the capacitance characteristics include capacitance intensity and capacitance change rate. The step of determining an operable instruction region as the target intent from multiple operable instruction regions based on eye movement and capacitance features corresponding to those regions includes: The dwell time, number of times the instruction is viewed, capacitance intensity, and capacitance change rate of each operable instruction area are normalized respectively. For each operable instruction region, the intent score corresponding to that operable instruction region is determined based on the normalized dwell time, normalized number of gazes, normalized capacitance intensity, and normalized capacitance change rate. The operable instruction region with the highest intent score among the operable instruction regions is taken as the target intent.

6. The intent recognition method as described in claim 1, characterized in that, The pop-up reminder on the touchscreen based on preset standard process information and the target intent includes: Standard operations are determined based on the preset standard process information; If the operation instruction corresponding to the target intent conforms to the standard operation, a confirmation pop-up reminder will appear next to the target intent area on the touch screen; the confirmation pop-up reminder is used to provide positive feedback on the operator's operation intent; If the operation instruction corresponding to the target intent does not conform to the standard operation, a guidance pop-up window will appear next to the target intent area on the touch screen; the guidance pop-up window is used to guide the operator to perform the standard operation.

7. The intent recognition method as described in claim 1, characterized in that, Determining the operator's target intent region based on the first intent region and the second intent region includes: The overlapping area of ​​the first intent region and the second intent region is determined as the target intent region of the operator.

8. An intent recognition device, characterized in that, A controller is used in an elevator teaching simulation system, which further includes a touchscreen, a near-field sensing module, and an eye-tracking recognition module. The near-field sensing module is used to sense capacitance changes within a preset range outside the touchscreen, and the eye-tracking recognition module is used to acquire changes in the operator's eye movements. The intention recognition device includes: The data acquisition module is used to acquire capacitance information sent by the near-field sensing module and eye movement information sent by the eye movement recognition module; the capacitance information includes capacitance change regions and capacitance change information corresponding to the capacitance change regions; the eye movement information includes eye movement trajectory regions and dwell time corresponding to the eye movement trajectory regions. The first intent recognition module is used to determine the first intent region of the operator based on the capacitance change region and the capacitance change information. The second intent recognition module is used to determine the operator's second intent region based on the eye movement trajectory region and the dwell time; The third intent recognition module is used to determine the target intent region of the operator based on the first intent region and the second intent region; The intent recognition and pop-up module is used to recognize the operator's intent based on the target intent area, obtain the target intent, and display a pop-up reminder on the touch screen based on preset standard process information and the target intent; wherein, the target intent is used to characterize the operator's intended operation, and the pop-up reminder is used to guide the operator's subsequent operations.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images