Interactive Control Method and System for Self-Service Borrowing Terminal Based on Transparent Touch Screen

Abstract

This invention discloses an interactive control method and system for a self-service borrowing terminal based on a transparent touchscreen, relating to the field of intelligent library self-service borrowing and human-computer interaction control technology. It addresses the problem of unstable mapping of touch coordinates to book selection positions on physical bookshelves when a transparent touchscreen covers the physical bookshelf, leading to borrowing and returning anomalies. The terminal is activated by touch or voice within a single interaction session, unifying the touch context identifier and speech recognition text. When the intent is to borrow or search, a mapping from screen coordinates to bookshelf coordinates is established based on an initial calibration homography matrix. The session state matrix is ​​recursively updated using touch discreteness, migration amount, correction indication amount, and intent uncertainty, combined with stability judgment to solidify the target book shelf identifier. Finally, the data is written to the business server and output via screen display and voice feedback.

Description

Technical Field

[0001] This invention relates to the field of intelligent library self-service borrowing and human-computer interaction control technology, and more specifically, to a self-service borrowing terminal interaction control method and system based on a transparent touch screen. Background Technology

[0002] Self-service borrowing machines in public spaces often use touchscreens as the entry interface. However, when a transparent touchscreen covers a physical bookshelf for a "what you see is what you select" interaction, the touch coordinates originate from the screen plane, while the user's actual intent corresponds to a shelf or position on the bookshelf plane. Due to differences in installation angles and distances at different locations, offsets after transportation and maintenance, and parallax and landing point differences among users of different heights, relying solely on fixed coordinates or touch context identifiers can easily lead to anomalies where the user sees point A but the system interprets it as point B. This results in problems such as failed book borrowing location, unresponsive repeated touches, and poor consistency across locations. Existing solutions often rely on one-time calibration during deployment or external rule correction, lacking a mechanism to dynamically update the mapping relationship based on touch behavior and voice error correction signals within the same interactive session. Furthermore, the borrowing and returning process requires a collaborative closed loop with reader identity verification, borrowing permissions, confirmation of actions and statuses of executing agencies, business data entry, and feedback presentation. Without unified session parameter fixing and traceable logs, reconciliation and recovery are difficult in the event of communication anomalies or misoperations.

[0003] To address the above problems, this invention proposes a solution. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a self-service borrowing terminal interactive control method and system based on a transparent touch screen to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] In a preferred embodiment, it includes:

[0007] After a touch event or voice wake-up is triggered, the self-service borrowing terminal generates an interactive session identifier and writes the touch context identifier or voice context text into the interactive session parameters. Then, it determines the operation intention based on a multi-class discrimination model and writes it into the interactive session parameters. When the operation intention is to borrow books or search for books, the self-service borrowing terminal saves the initial calibration data in advance based on at least four sets of calibration points collected during the installation and debugging phase and obtains the initial homography matrix as the initial mapping relationship. Within the interactive session, it maintains the session-state homography matrix used to map the screen touch coordinates to the bookshelf plane coordinates and updates it recursively.

[0008] Within a session, the terminal constructs touch point distribution parameters from the touch event sequence to obtain touch dispersion and touch center migration, constructs correction indication from the voice context text, and constructs intent uncertainty from the intent category probability distribution. Then, based on touch dispersion, migration, correction indication, and intent uncertainty, it calculates update coefficients to adjust the update step size and update the session homography matrix.

[0009] The terminal performs spatial matching between the bookshelf plane coordinates obtained by mapping the current touch coordinates and the boundaries of the preset book area to obtain the candidate book area number. When the candidate book area number belongs to the preset candidate book sequence set and the touch dispersion meets the stability judgment threshold, the target book identifier is determined and fixed and written into the interaction session parameters. When the correction indication meets the preset conditions and the migration exceeds the threshold, the terminal presents at least four guide touch points on the transparent touch screen and collects the corresponding point pairs to update the session homography matrix.

[0010] The self-service borrowing terminal collects RFID media identifiers and facial images, completes identity verification, and generates borrowing permissions, which are then written into the interactive session parameters.

[0011] The self-service borrowing terminal controls the book retrieval mechanism to execute the borrowing or returning of books based on the user's intention and borrowing permissions, and writes the status confirmation result.

[0012] The borrowing and returning results are written to the library's business server interface and the local transaction log, and presented via a transparent touchscreen or voice broadcast.

[0013] In a preferred embodiment, the self-service borrowing terminal displays the main interface on a transparent touch screen and simultaneously activates the touch controller and array microphone as input sources. When a touch event is detected or a voice segment that meets the preset wake-up conditions is detected, an interaction session identifier is generated to identify the current interaction session and the start time of the interaction session is recorded.

[0014] Within the same interactive session, the self-service borrowing terminal reads the touch context identifier corresponding to the touch event and records the touch context identifier as the touch context identifier. It then performs speech recognition on the voice signal to obtain the voice context text.

[0015] In a preferred embodiment, the self-service borrowing terminal generates a fusion feature for intent discrimination based on the voice context text and the touch context identifier, and outputs the probability distribution of intent categories through a multi-class discrimination model to determine the operation intent. The operation intent is written into the interaction session parameters, and after the operation intent is determined, the matching target determination process is selected and executed according to the operation intent, so that the interaction session parameters are fixed when the interaction session ends.

[0016] In a preferred embodiment, once the operation intent in the interactive session parameters is determined, the self-service borrowing terminal performs target determination and permission generation according to the operation intent: if it is for returning books, it switches the book return guidance interface, reads the book return entrance ready status given by the door lock status signal or entrance status sensor signal, and writes it into the interactive session parameters; if it is for reading e-books, it displays the e-book catalog, receives touch selection to determine the target e-book item, and writes the unique identifier of the item into the interactive session parameters; if it is for voice assistance, it keeps the voice interaction interface active and writes the voice context text and its recognition timestamp into the interactive session parameters.

[0017] Within the same interactive session, the self-service borrowing terminal activates the RFID reader and image acquisition device to obtain the RFID medium identifier and the reader's face image and writes them into the interactive session parameters. Subsequently, it performs face detection and key point alignment on the face image and generates a face feature vector through a face feature extraction model. Based on the RFID medium identifier, it retrieves the bound face feature vector and reader account status information from the library's reader database. By calculating the cosine similarity and comparing it with a preset similarity threshold, it obtains the face verification result. At the same time, it outputs the RFID medium identifier verification result and the account status verification result and writes them into the interactive session parameters. Finally, based on the operation intention and each verification result, it generates borrowing permissions and writes them into the interactive session parameters. When generating the allowed borrowing permissions, it reads the maximum number of books that can be borrowed and the number of books that have been borrowed from the library's reader database to calculate the remaining borrowing capacity and uses it as a constraint condition for the borrowing permissions.

[0018] In a preferred embodiment, the self-service borrowing terminal controls the borrowing and returning of books and writes the status confirmation result according to the operation intention and borrowing permission in the interactive session parameters: when the operation intention is to borrow books, the terminal first determines whether to start the borrowing operation based on the remaining borrowing quantity. After starting, it reads the target book grid identifier to determine the location coordinates of the target book grid and obtains the target book's RFID tag identifier through RFID reading and matching, and writes it into the interactive session parameters. Then, it controls the book retrieval mechanism to reach the target position, pick up the book, and transfer it to the book outlet. It controls the entrance door lock actuator to unlock and sends out the book after the door lock status signal confirms the unlocking. The book outlet photoelectric sensor confirms the book's arrival and departure and writes the arrival status and taken-away confirmation results respectively.

[0019] Subsequently, the RFID tag of the book read from the book outlet is compared with the RFID tag of the target book, and the comparison result is written into the interactive session parameters. If there is a discrepancy, the book is retrieved. When the operation intention is to return a book, the terminal first determines whether the borrowing permission allows the book to be returned. If the return is allowed, the book is allowed to enter the return entrance based on the readiness status of the return entrance and the door lock status signal. After confirming the location with a photoelectric sensor, the RFID tag of the book is read as the unique identifier of the returned object and written into the interactive session parameters. Then, the storage location coordinates are determined according to the book compartment location coordinate mapping table and the inventory strategy, and the book retrieval execution mechanism is controlled to complete the storage. The location feedback and the status of the book compartment entrance are used to confirm the placement is completed and the book storage confirmation result is written into the interactive session parameters, and the door lock of the return entrance is closed. When the operation intention is to read e-books, search for books, or provide voice assistance, the terminal performs content loading, collection retrieval, or semantic matching based on the unique identifier of the target e-book, the touch context identifier, or the voice context text in the interactive session parameters, and writes the corresponding result generation status into the interactive session parameters.

[0020] In a preferred embodiment, the self-service borrowing terminal performs result writing and feedback output based on the borrowing and returning confirmation results in the written interaction session parameters: when the borrowing confirmation result is that the book has been taken, the terminal constructs a borrowing write request using the RFID medium identifier and the RFID tag identifier of the target book, sends it to the library business server interface, receives the write response, writes the write status code and borrowing record identifier into the interaction session parameters and records them in the local transaction log. If the write is successful, the on-shelf status of the collection index is updated and the current number of borrowed books is updated; if the write fails, the failure status and reason code are written. When the returning confirmation result is that the book has been put away, the terminal... The system constructs a return write request using RFID media identifiers and book RFID tag identifiers, and writes a status code and return record identifier. If the write is successful, it updates the shelf status of the collection index and writes the updated current number of borrowed books. If the write fails, it writes a failure status and reason code. The terminal then outputs the write result via a transparent touch screen or voice synthesis broadcast. At the end of the interaction session, it saves the interaction session parameters to the local transaction log and uploads the write event summary for consistency reconciliation when the network is normal. At the same time, it clears temporary data related to intent identification and only retains the write result and local transaction log record necessary for the borrowing and returning business.

[0021] In a preferred embodiment, it includes: a session intent mapping module, an identity verification and authorization module, a borrowing and returning execution confirmation module, a result writing feedback module, and signal connections between the modules;

[0022] The session intent mapping module is used by the self-service borrowing terminal to generate an interactive session identifier after a touch event or voice wake-up is triggered, and write the touch context identifier or voice context text into the interactive session parameters. Then, it determines the operation intent based on a multi-class discrimination model and writes it into the interactive session parameters. When the operation intent is to borrow books or search for books, the self-service borrowing terminal saves the initial calibration data in advance based on at least four sets of calibration points collected during the installation and debugging phase and obtains the initial homography matrix as the initial mapping relationship. Within the interactive session, it maintains the session state homography matrix used to map the screen touch coordinates to the bookshelf plane coordinates and updates it recursively.

[0023] Within a session, the terminal constructs touch point distribution parameters from the touch event sequence to obtain touch dispersion and touch center migration, constructs correction indication from the voice context text, and constructs intent uncertainty from the intent category probability distribution. Then, based on touch dispersion, migration, correction indication, and intent uncertainty, it calculates update coefficients to adjust the update step size and update the session homography matrix.

[0024] The terminal performs spatial matching between the bookshelf plane coordinates obtained by mapping the current touch coordinates and the boundaries of the preset book area to obtain the candidate book area number. When the candidate book area number belongs to the preset candidate book sequence set and the touch dispersion meets the stability judgment threshold, the target book identifier is determined and fixed and written into the interaction session parameters. When the correction indication meets the preset conditions and the migration exceeds the threshold, the terminal presents at least four guide touch points on the transparent touch screen and collects the corresponding point pairs to update the session homography matrix.

[0025] The identity verification and authorization module is used by the self-service borrowing terminal to collect RFID media identifiers and facial images, complete identity verification, and generate borrowing permissions and write them into the interactive session parameters.

[0026] The borrowing and returning execution confirmation module is used by the self-service borrowing terminal to control the book retrieval execution mechanism to execute the borrowing or returning of books according to the operation intention and borrowing permission, and write the status confirmation result.

[0027] The results are written to the feedback module, which writes the borrowing and returning results to the library business server interface and to the local transaction log, and presents them on a transparent touch screen or through voice broadcast.

[0028] The technical effects and advantages of the self-service borrowing terminal interactive control method and system based on a transparent touch screen of this invention are as follows:

[0029] This invention integrates touch, voice, location, and borrowing / returning into a single interactive session parameter system: Given a clear intent, screen-to-bookshelf one-way mapping is activated only when borrowing or querying is triggered. The update step size is jointly adjusted using touch clustering, center migration, voice correction signals, and intent entropy, enabling the mapping to adaptively converge with installation offsets, user differences, and short-term drift. Furthermore, candidate shelf matching and continuous touch stability assessment significantly reduce misselection and jitter. Combining RFID and facial recognition for permission generation, the actions of the execution mechanism are confirmed by sensors in a closed loop and written to the transaction log, ensuring traceability and reconciliation of borrowing and returning records. Multi-channel feedback enhances availability and fault tolerance, thereby improving cross-site consistency, location reliability, and business success rate. Attached Figure Description

[0030] Figure 1 This is a timing diagram of the interactive control method and system of the self-service borrowing terminal based on a transparent touch screen according to the present invention.

[0031] Figure 2 This is a schematic diagram of the interactive control method and system modules of the self-service borrowing terminal based on a transparent touch screen according to the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Example

[0034] This invention discloses an interactive control method for a self-service borrowing terminal based on a transparent touchscreen, such as... Figure 1 As shown, it includes:

[0035] Step 1: Interactive wake-up and acquisition of operation intent;

[0036] In step one, the self-service borrowing terminal based on the transparent touch screen completes the interaction wake-up, input normalization and operation intent discrimination within one interaction session, and completes the target determination process that matches the operation intent after the operation intent is determined, so that a set of solidified interaction session parameters are formed at the end of the interaction session.

[0037] First, the self-service borrowing terminal displays its main interface on a transparent touchscreen. This main interface includes at least a book borrowing / returning entry, an AI voice assistant entry, and an e-book entry. The self-service borrowing terminal simultaneously activates two input sources: the first is the output of the touch controller on the transparent touchscreen, including at least touch coordinates and a touch timestamp; the second is the voice signal collected by the array microphone, including at least voice waveform data and a collection timestamp. The self-service borrowing terminal uses either input source as the interaction wake-up condition: when a touch event is detected by the touch controller, or when the array microphone collects a voice segment that meets a preset wake-up word condition or a preset energy threshold condition, the self-service borrowing terminal generates an interaction session identifier and records the interaction session start time; the interaction session identifier is a random or time-series unique value used to identify this interaction session.

[0038] Within the same interactive session, the self-service borrowing terminal performs unified input normalization processing on touch input and voice input to form context data that can be used for the same intent discrimination model. For touch input, the self-service borrowing terminal reads the touch context identifier corresponding to the touch event and records this touch context identifier as the touch context identifier; the touch context identifier includes at least a book borrowing or returning entry identifier, an AI voice assistant entry identifier, or an e-book entry identifier. For voice input, the self-service borrowing terminal performs speech recognition processing on the voice waveform data to obtain the voice context text; the speech recognition processing is completed by a local speech recognition engine or a cloud speech recognition interface, and the voice context text corresponds to the recognition result of the current voice segment. After completing input normalization, the self-service borrowing terminal writes the touch context identifier, the voice context text, and their respective timestamps into the interactive session parameters.

[0039] Subsequently, the self-service borrowing terminal performs operation intent determination within the same interactive session. The self-service borrowing terminal calculates the operation intent using a multi-class discriminant model based on Softmax, outputting the probability distribution of each intent category, satisfying the following relationship:

[0040] ;

[0041] Where k represents the candidate intent category index, and K represents the total number of intent categories; Represents the fused feature vector; This represents the parameter vector corresponding to the k-th type of intent. The fused feature vector... It consists of two parts: one is the text features obtained by vectorizing the speech context text, and the other is the touch features obtained by encoding the touch context identifier;

[0042] In a preferred embodiment, the vectorization of the speech context text is as follows: after performing word segmentation and stop word removal on the speech context text, text features are generated according to the keyword vocabulary preset in the terminal; wherein, the keyword vocabulary in this solution is a keyword set for the operation intention discrimination task, and the keyword vocabulary is固化 during device deployment to ensure consistency and reproducibility across different network points. Preferably, the keyword vocabulary is constructed by merging high-frequency word entries statistically obtained from the speech context text in historical interaction sessions and intention-related word entries preset in the service, and synonym word entries are merged and low-frequency word entries are removed to form the final vocabulary; the number of word entries in the keyword vocabulary is denoted as N, N is a preset positive integer, preferably 200 to 3000. For the convenience of those skilled in the art to reproduce, the keyword vocabulary may at least include the following example word entries: borrow books, borrow, I want to borrow, get books, take books, borrow one, borrow this, related to borrowing books; return books, return, I want to return, put back, return it, related to returning books; query, find books, search for books, is there, where, library collection, location, book title, author, call number, related to book query; e-books, e-books, reading, open, table of contents, chapter, related to e-book reading; help, how to use, can't use, hint, guidance, instructions, related to voice assistance. Preferably, the stop words at least include functional words or colloquial word entries that contribute less to intention discrimination, such as "of", "already", "in", "for a moment", "help me", "excuse me", etc.

[0043] Furthermore, the text feature is a fixed-dimensional vector, and its dimension is equal to the number of word entries N in the keyword vocabulary. Preferably, the text feature is generated using a count vector based on the bag of words: the self-service borrowing terminal represents the speech context text after word segmentation and stop word removal as a sequence of word tokens, and constructs an N-dimensional vector in the fixed order of the keyword vocabulary , where ti represents the number of occurrences of the speech context text that matches the i-th keyword entry; when it does not appear, ti = 0. To reduce the impact of the difference in sentence length, preferably, the self-service borrowing terminal performs normalization processing on the to obtain the text feature: when || || ≠ 0, let ← || ||; when || || = 0, it remains a zero vector.

[0044] The encoding of the touch context identifier is as follows: using at least the borrowing or returning book entry identifier, the AI voice assistant entry identifier, or the e-book entry identifier included in the touch context identifier as the encoding dimension, the number of encoding dimensions is denoted as T, T is a preset positive integer; let the corresponding dimension of the current touch context identifier take 1, and the remaining dimensions take 0 to obtain the touch feature; correspondingly, the self-service borrowing terminal constructs a T-dimensional vector =[c1,c2,…,cT]ᵀ, where cj corresponding to the current touch context identifier is 1, and the rest are 0.

[0045] The self-service borrowing terminal concatenates the text features and the touch features in a fixed order to obtain a fused feature vector x=[ , ]ᵀ, wherein the dimension of the fused feature vector is (N+T). For example, when N=1000 and T=4, the fused feature vector is a 1004-dimensional vector; when the speech context text contains borrowing or taking a book and the touch context identifier is the borrowing entry identifier in the borrowing or returning entry identifier, the relevant dimension of the corresponding text feature is 1 and the touch feature is [1,0,0,0]ᵀ.

[0046] The self-service borrowing terminal inputs the fused feature vector into a multi-class discrimination model based on Softmax to output the probability distribution of the intent category, and determines the operation intent based on the intent category corresponding to the highest probability.

[0047] Furthermore, in this scheme, the parameter vector of the multi-class discrimination model is the model parameter that is fixed during device deployment. The model parameter can be obtained by offline training using training samples extracted from historical interaction session parameters, and after verification before the model goes online, it is written into the local model file of the terminal or sent to the terminal by the business server for use.

[0048] Both the text features and touch features are derived from context data already written within the current interaction session. The self-service borrowing terminal uses... The corresponding intent category determines the operation intent, which includes at least borrowing books, returning books, searching for books, reading e-books, and voice assistance. The determined operation intent is then written into the interaction session parameters.

[0049] After the operation intention is written, the self-service borrowing terminal continues to execute the operation intention within the same interactive session. Figure 1 The target determination process is driven by operational intent, and its processing path is determined by external descriptive rules.

[0050] It should be noted that when deploying a transparent touchscreen self-service borrowing terminal in a public space, the transparent touchscreen simultaneously serves as both the information presentation interface and the touch input interface. The terminal displays borrowing / returning entry points, an AI voice assistant entry point, and an e-book entry point on the main interface. The terminal uses the touch controller as the primary input source and the array microphone as the secondary input source. Upon touch event or voice wake-up, it generates an interaction session identifier, writes it into the touch context identifier or voice context text, and then uses a Softmax-based multi-class discriminant model to calculate the user's intent and write it into the interaction session parameters. At this point, the terminal can reliably answer what the user wants to do and can solidify the user's touch- or voice-triggered actions within the same interaction session.

[0051] However, when the user's intention is determined to be borrowing or searching for books, a contradiction emerges that is only noticeable when the transparent touchscreen covers the physical bookshelf: the coordinates of the user's touch on the transparent touchscreen come from the screen plane coordinates output by the touch controller, while the user actually intends to select a specific location or shelf area on the bookshelf plane inside the cabinet. More importantly, the installation angle and location of the transparent touchscreen may vary across different locations, and new offsets may occur after equipment transportation, relocation, maintenance, or replacement of the transparent touchscreen. In this situation, even if the user's intention has been determined by the Softmax discrimination model, the self-service borrowing terminal still cannot reliably interpret the user's touch action as a selection result on the bookshelf plane inside the cabinet based solely on the touch coordinates and touch context identifiers. This directly leads to a discrepancy between the area seen by the touch and the area understood by the terminal, creating a recurring interactive anomaly: the same user may touch the same transparent touchscreen area repeatedly at the same location, but the terminal may still produce different selection and understanding results at different deployment points; at the same time, users of different heights may have the same visual direction for the same physical book, but the landing point on the touch plane is different, causing the phenomenon of consistent visual direction but inconsistent touch coordinates to occur repeatedly during high-frequency use.

[0052] Therefore, in this embodiment, when the operation intent is to borrow a book or search for a book, the self-service borrowing terminal interprets the screen touch coordinates output by the touch controller of the transparent touchscreen as the selection position on the bookshelf plane inside the cabinet, and determines the target book shelf identifier accordingly. The self-service borrowing terminal pre-stores the initial calibration data of the screen coordinate system and the bookshelf coordinate system. The initial calibration data is obtained from the calibration point pairs collected during the installation and debugging phase. The calibration point pairs contain at least four sets, which are the screen touch point coordinates. Coordinates of the point corresponding to the plane of the bookshelf The self-service borrowing terminal obtains the initial homography matrix based on the initial calibration data. and the initial homography matrix Used as the initial value for the mapping relationship.

[0053] It should be noted that the touch coordinates output by the touch controller may shift or drift depending on the reader, time, and device state. To ensure that the mapping relationship can be updated within the interactive session, the self-service borrowing terminal maintains session-state parameters of the homography matrix within the interactive session. and session state parameters Perform a recursive update; where the session-state parameters In this scheme, it is defined as: the current homography matrix used to map screen touch coordinates to bookshelf plane coordinates, and its initial value is taken as... And it is updated based on the in-session observables during the interaction session. For any touch event, let the screen touch coordinates output by the touch controller be... Then the plane coordinates of the bookshelf It is given by the following formula:

[0054] ;

[0055] Where s is the scaling factor, These are session-state parameters.

[0056] The self-service borrowing terminal constructs touch point distribution parameters based on the touch event sequence within the interactive session. In this scheme, the touch point distribution parameters are defined as: a statistic calculated from the set of touch coordinate samples within the most recent time window, used to characterize the clustering degree and migration trend of touch points. Let the set of touch coordinate samples collected within the most recent time window be... ,in If the touch timestamp is used, the self-service borrowing terminal calculates the coordinates of the touch center. With the touch dispersion D, the following is satisfied:

[0057] ;

[0058] ;

[0059] ;

[0060] The self-service borrowing terminal also calculates the migration amount M of the touch center coordinates between adjacent windows. In this scheme, the migration amount is defined as the distance between the touch center of the current window and the touch center of the previous window, satisfying:

[0061] ;

[0062] in, The coordinates are the touch center coordinates of the previous window.

[0063] Furthermore, the self-service borrowing terminal constructs a correction indication quantity for the voice context text within the interactive session. In this scheme, the correction indication quantity is defined as: a quantified result determined by whether a preset set of correction statements appears in the voice context text; the preset set of correction statements is defined in this scheme as: a set of statement patterns used to indicate negation or correction, including at least the statement patterns such as "It's not this book I clicked the wrong one, I clicked this book." Let the most recent segment of voice context text within the interactive session be... Then when C=1 is set when any preset correction statement pattern is matched, otherwise C=0 is set, and C is used as the correction indicator.

[0064] The self-service borrowing terminal constructs intent uncertainty based on the probability distribution of each intent category in the Softmax output during the interactive session. Let the probability distribution of the Softmax output be... ,in To fuse feature vectors, and given K as the total number of intent categories, the intent uncertainty Q is calculated using the following formula:

[0065] ;

[0066] in, This is the operation for the natural logarithm.

[0067] After obtaining the touch dispersion D, migration M, correction indication C, and intent uncertainty Q, the self-service borrowing terminal sets the session state parameters. Perform a recursive update. The self-service borrowing terminal will update the session parameters. Expand into a parameter vector by row The parameter vector In this scheme, it is defined as: session state parameters A vector composed of free parameters in a fixed order after removing the proportional invariance.

[0068] Preferably, the self-service borrowing terminal uses fixed session parameters. The third row and third column element is 1 to eliminate the proportional invariance; correspondingly, the parameter vector θ is taken in a fixed order of rows as: [h11,h12,h13,h21,h22,h23,h31,h32]ᵀ, where hij is the i-th row and j-th column element of the session state parameter H.

[0069] The self-service borrowing terminal calculates and updates the coefficient γ within the interactive session, satisfying:

[0070] ;

[0071] Where σ(·) is an S-shaped function, η is a preset coefficient, and γ is an update coefficient with a value range of (0,1). , , , Here, is a preset coefficient, and e is a natural constant. Subsequently, the self-service borrowing terminal will set the current touch coordinates... Session parameters Mapping to obtain the plane coordinates of the bookshelf and will Spatial matching with the boundaries of the preset book area is performed to obtain the candidate book area number. The pre-set shelf area boundary is defined in this scheme as: the set of polygonal boundaries of each shelf in the bookshelf coordinate system, which is measured and stored during the equipment deployment phase. The self-service borrowing terminal reads the candidate shelf area number. Reference center point of the corresponding book grid And construct the residual vector r, satisfying:

[0072]

[0073] Among them, the reference center point In this scheme, it is defined as: the centroid or preset center point calculated from the polygon boundary corresponding to the candidate book grid. The self-service borrowing terminal calculates the mapping function with respect to the parameter vector. Jacobian matrix The Jacobian matrix In this scheme, the coordinates of the bookshelf plane are defined as follows: For parameter vectors A matrix composed of the first-order partial derivatives.

[0074] in, For the session state parameters The self-service borrowing terminal uses the Euclidean coordinates obtained by aligning the normalized projection of the secondary coordinates when calculating the Jacobian matrix J. As the object of differentiation.

[0075] To avoid the introduction of incorrect reference center points due to large initial mapping errors or sudden environmental changes, which could lead to divergence in recursive updates, the self-service borrowing terminal preferably performs a reliability determination on the candidate book grid area numbers before reading the reference center point of the book grid corresponding to the candidate book grid area number and constructing the residual vector r. The reliability determination includes at least one of the following conditions:

[0076] One is the boundary constraint condition: the self-service borrowing terminal determines whether the current bookshelf plane coordinates are located inside the polygon boundary corresponding to the candidate book grid; when it is located inside the polygon boundary and the minimum distance to the polygon boundary is not less than the preset boundary distance threshold, the candidate book grid area number is determined to be reliable; when it is located outside the polygon boundary or the minimum distance to the boundary is less than the preset boundary distance threshold, the candidate book grid area number is determined to be unreliable.

[0077] The second condition is sequence consistency: The self-service borrowing terminal statistically analyzes the candidate library area number sequence of the most recent L touch events. If the mode frequency of this sequence is not less than a preset frequency threshold, the candidate library area number is considered reliable; if the mode frequency is less than the preset frequency threshold, the candidate library area number is considered unreliable.

[0078] When the candidate book grid area number is unreliable, the self-service borrowing terminal preferably does not perform parameter vector θ update based on residual vector r and Jacobian matrix, but maintains the current session state parameters for subsequent touch event mapping calculation, and continues to collect subsequent touch events to wait for the stability judgment to be met; during this period, if the triggering condition for in-session relocation processing is met, the self-service borrowing terminal performs in-session relocation processing.

[0079] The self-service borrowing terminal adjusts the step size using the update coefficient γ and updates the parameter vector using the following formula:

[0080] ;

[0081] Where α is the preset base step size. The self-service borrowing terminal will update the parameter vector. Restore to session state parameters and update the session state parameters Used for mapping calculations of subsequent touch events.

[0082] To suppress update divergence caused by erroneous reference centers, the self-service borrowing terminal preferably implements residual constraints and update amount constraints before updating the parameter vector θ: Let the norm threshold of the residual vector r be a preset residual threshold. When ||r|| is greater than the preset residual threshold, the self-service borrowing terminal does not perform the current parameter vector θ update and triggers in-session relocation processing or keeps the current session parameters unchanged; when ||r|| is not greater than the preset residual threshold, the self-service borrowing terminal allows the parameter vector θ update. Furthermore, the self-service borrowing terminal can set a preset update amount threshold for the parameter increment of a single update. When the calculated parameter increment norm exceeds the preset update amount threshold, the self-service borrowing terminal limits the update magnitude by reducing the preset base step size α or setting the update coefficient γ to a smaller value.

[0083] Furthermore, to ensure the convergence of the recursive update process, the self-service borrowing terminal preferably maintains the session state parameters corresponding to the most recent stability determination as stable session state parameters H_stable, and maintains residual value to characterize the update effect. After each update of the session state parameters, the self-service borrowing terminal recalculates the residual vector r based on the updated session state parameters and obtains the updated residual value; when the residual value does not decrease for P consecutive times or increases for P consecutive times, it is determined that update divergence has occurred. When update divergence is determined, the self-service borrowing terminal reverts the session state parameters to the stable session state parameters H_stable and triggers intra-session relocation processing to re-acquire and guide the touch point to update the session state parameters.

[0084] To avoid the instability caused by short-term abnormal touches to the target book grid identifier, the self-service borrowing terminal performs a stability determination within the interactive session and solidifies the target book grid identifier when the stability determination conditions are met. The stability determination is defined in this scheme as: a process of determining whether the candidate book grid area number sequence meets a concentration condition and the touch dispersion meets a threshold condition within a preset number of consecutive touches. Specifically, the self-service borrowing terminal counts the candidate book grid area number sequence of the most recent L touch events. When the mode of the sequence is And the frequency of the mode is not less than the preset frequency threshold. And the touch dispersion within the corresponding time window At that time, the self-service borrowing terminal will identify the target book compartment as... And write the target cell identifier into the interaction session parameters; where, To preset the dispersion threshold, This is a preset frequency threshold.

[0085] In a preferred embodiment, the preset time window, L, preset frequency threshold, and preset dispersion threshold are all system configuration parameters and are set during device deployment. The preset time window is preferably 1s to 3s, used to limit the statistical sampling range of the most recent L touch events; L is preferably 6 to 15, used to balance interaction response speed and stability. The preset frequency threshold is preferably not less than ⌈0.6L⌉ and not greater than L, to ensure sufficient concentration of the candidate book area number sequence. The preset dispersion threshold is determined as follows: during device deployment, multiple readers of different heights are organized to continuously sample touches for the same book area number, the statistical mean of the touch dispersion D is calculated, and the preset dispersion threshold is set to 1.5 to 3 times the statistical mean to cover the normal touch jitter range but exclude abnormal touch dispersion.

[0086] Furthermore, the preset base step size α is a system configuration parameter set during device deployment, preferably 0.05~1.00, used to adjust the recursive update amplitude of the session state parameters; the preset coefficient is a system configuration parameter set during device deployment, preferably 1~10, used to adjust the sensitivity of the update coefficient γ. The preset energy threshold condition is a system configuration parameter set during device deployment, and its setting method is: collecting environmental noise at the deployment point in the terminal standby state to form an energy baseline, and adding a fixed increment to the energy baseline as the voice wake-up trigger threshold. The preset hold time is a system configuration parameter set during device deployment, preferably 0.3s~2.0s, used to suppress misjudgments caused by momentary jitter when the photoelectric sensor detects that the book has left the device.

[0087] Furthermore, when the correction indication satisfies C=1 and the migration amount is satisfied within a preset time window, or when the norm of the residual vector r is greater than a preset residual threshold, or when the mode frequency of the candidate shelf area number sequence of the most recent L touch events is less than a preset frequency threshold, or when an update divergence is determined, the self-service borrowing terminal performs in-session relocation processing. The in-session relocation processing is defined in this scheme as: a process of presenting guide touch points on a transparent touchscreen and updating session state parameters using guide touch input without exiting the interactive session. Specifically, the self-service borrowing terminal presents at least four guide touch points on the transparent touchscreen. These guide touch points are defined in this scheme as: screen display positions corresponding one-to-one with the visible reference positions of the bookshelf; the self-service borrowing terminal records the touch coordinates when the reader sequentially touches the guide touch points. And read the coordinates of the corresponding bookshelf plane reference point. The coordinates of the bookshelf plane reference point are defined in this scheme as: the coordinate values ​​in the reference point coordinate table fixed during the equipment deployment phase. The self-service borrowing terminal is based on a point-to-point set. Update session parameters Minimize the following objective function:

[0088] ;

[0089] in, This indicates the operation of normalized projection based on the aligned secondary coordinates. After the intra-session relocation process is completed, the self-service borrowing terminal uses the updated session state parameters. Continue performing the mapping calculation from touch coordinates to bookshelf plane coordinates, and solidify and write the target shelf identifier according to the stability determination.

[0090] When the intention is to return a book, the self-service borrowing terminal switches to the book return guidance interface on the transparent touch screen and reads the readiness status of the book return entrance. The readiness status is given by the door lock status signal or entrance status sensor signal at the book return entrance. The self-service borrowing terminal writes the readiness status into the interactive session parameters and presents it on the interface as a status prompt.

[0091] When the intended operation is e-book reading, the self-service borrowing terminal displays an e-book catalog on a transparent touchscreen and receives touch selections to determine the target e-book item. The self-service borrowing terminal reads the unique identifier of the target e-book item and writes it into the interactive session parameters; the unique identifier comes from the terminal's pre-set e-book resource index table or the result returned by the cloud resource catalog.

[0092] When the intended action is voice assistance, the self-service borrowing terminal keeps the voice interaction interface active and records the voice context text and its recognition timestamp as part of the interaction session parameters; the voice context text comes from the output of the voice recognition processing.

[0093] Step Two: Identity Verification and Borrowing Permission Generation;

[0094] In step two, the self-service borrowing terminal based on the transparent touch screen completes the acquisition of reader identity, verification of reader identity, and generation of borrowing permissions based on the interactive session parameters formed in step one, and writes the identity verification result and borrowing permission result into the interactive session parameters.

[0095] Specifically, the self-service borrowing terminal is pre-configured and enabled with a reader identification device, which includes at least an RFID reader and an image acquisition device. The RFID reader reads the reader's RFID tag, which originates from the reader's library card or campus ID, etc.; the RFID tag is a unique identifier for the RFID tag. The image acquisition device captures the reader's facial image; the image acquisition device is a camera positioned above or near the transparent touchscreen. During the interactive session, the self-service borrowing terminal simultaneously initiates the acquisition process of both the RFID reader and the image acquisition device: the RFID reader reads the RFID tag within a preset reading distance and outputs a reading timestamp; the image acquisition device captures an image of the reader's facial area at a preset frame rate and outputs an image acquisition timestamp. The self-service borrowing terminal writes the RFID tag, the facial image, and their corresponding timestamps into the interactive session parameters to form the raw data required for identity verification.

[0096] After writing the original data, the self-service borrowing terminal performs face image detection and alignment processing to obtain a face feature vector for identity verification. Specifically, the self-service borrowing terminal performs face detection processing on the collected face image, outputting a face bounding box and facial key points, wherein the facial key points include at least the center points of the eyes, the tip of the nose, and the corners of the mouth. Based on the facial key points, the self-service borrowing terminal performs affine transformation alignment processing to normalize the face image to a preset size and preset pose, obtaining an aligned face image. In this scheme, the aligned face image is defined as: the image obtained by mapping the face region to a preset face template coordinate system through an affine transformation determined by the facial key points. The self-service borrowing terminal inputs the aligned face image into a face feature extraction model and outputs a face feature vector; the face feature extraction model is a deep neural network model, and its output face feature vector is a fixed-dimensional vector.

[0097] In a preferred embodiment, the facial feature vector is normalized after output to ensure comparability of facial feature vectors under different lighting and pose conditions. The dimension of the fixed-dimensional vector can be any of 128, 256, or 512 dimensions and remains unchanged within the same device deployment cycle. The preset similarity threshold T is a system configuration parameter set during device deployment, preferably between 0.60 and 0.90. The preset similarity threshold T is set as follows: during the device deployment phase, multiple aligned facial images of the same reader are collected, and the statistical distribution of the corresponding cosine similarity S is calculated. Combined with the statistical distribution of cosine similarity S between different readers, a threshold that can simultaneously meet the requirements of false rejection rate and false acceptance rate is selected as the preset similarity threshold T and stored.

[0098] To clarify the similarity calculation method, the self-service borrowing terminal uses cosine similarity to calculate the comparison similarity, satisfying the following relationship:

[0099] ;

[0100] Where S represents cosine similarity, Indicates to Transpose into a row vector This represents the facial feature vector obtained from the aligned face image of this interactive session. This represents the facial feature vector associated with the RFID medium identifier. In this scheme, the facial feature vector associated with the RFID medium identifier is defined as: a facial feature vector extracted from the registered facial images of readers bound to the RFID medium identifier in the library's reader database using the same facial feature extraction model.

[0101] Further, the self-service borrowing terminal retrieves the reader account record in the library reader database according to the RFID medium identifier, and reads the face feature vector and the reader account status information bound to the RFID medium identifier from the reader account record. The reader account status information at least includes the borrowing permission status, the overdue fee status, and the overdue status. After the self-service borrowing terminal retrieves the reader account record, it performs face verification and determination: taking the face feature vector obtained from the current interaction session as , taking the face feature vector read from the reader account record as , calculating the cosine similarity S, and comparing the cosine similarity S with the preset similarity threshold T, where the preset similarity threshold T is a system configuration parameter and is set during device deployment; when , the self-service borrowing terminal determines that the face verification is passed, and when S < T, the self-service borrowing terminal determines that the face verification is not passed. The self-service borrowing terminal writes the face verification result into the interaction session parameters.

[0102] After writing the face verification result, the self-service borrowing terminal performs RFID medium identifier verification and account status verification. The RFID medium identifier verification is defined in this solution as: determining whether the RFID medium identifier exists in the valid reader set of the library reader database, and determining whether the account corresponding to the RFID medium identifier is in an active state. The account status verification is defined in this solution as: determining whether borrowing permission is allowed based on the reader account status information. The self-service borrowing terminal reads the borrowing permission status, the overdue fee status, and the overdue status in the reader account status information, and outputs the account status verification result according to the preset verification rules, where the preset verification rules at least include: when the overdue fee status is overdue or the overdue status is overdue, the account status verification result is not passed; when the overdue fee status is not overdue, the overdue status is not overdue, and the borrowing permission status is borrowable, the account status verification result is passed. The self-service borrowing terminal writes the RFID medium identifier verification result and the account status verification result into the interaction session parameters.

[0103] After the RFID media identifier verification result, facial recognition verification result, and account status verification result have all been written into the interactive session parameters, the self-service borrowing terminal generates borrowing permissions. In this scheme, borrowing permissions are defined as: a set of authorization results and constraints that characterize whether a reader is allowed to perform a borrowing or returning operation in this interactive session. The self-service borrowing terminal determines the authorization type of borrowing permissions based on the operation intent written in step one: when the operation intent is to borrow or search for books and the RFID media identifier verification, facial recognition verification, and account status verification are all passed, the self-service borrowing terminal generates borrowing permissions that allow borrowing; when the operation intent is to return books and the RFID media identifier verification and facial recognition verification are both passed, the self-service borrowing terminal generates borrowing permissions that allow returning the book; when any verification result is unsuccessful, the self-service borrowing terminal generates borrowing permissions that deny authorization. For permitted borrowing permissions, the self-service borrowing terminal reads borrowing constraint parameters from the library's reader database and writes them into the borrowing permissions. These borrowing constraint parameters include at least the maximum number of books that can be borrowed and the current number of books already borrowed; both the maximum number of books that can be borrowed and the current number of books already borrowed are derived from reader account records in the library's reader database and are not generated by the terminal on an ad-hoc basis. The self-service borrowing terminal calculates the remaining borrowable quantity based on the maximum number of books that can be borrowed and the current number of books already borrowed, and this remaining quantity satisfies the following conditions:

[0104] ;

[0105] Where R represents the available margin. This indicates the maximum number of books that can be borrowed. This indicates the current number of books borrowed. The self-service borrowing terminal writes R into the borrowing permissions as one of the constraints.

[0106] At the end of step two, the self-service borrowing terminal generates and solidifies the following results in the interactive session parameters: RFID medium identifier, facial feature vector, facial verification result, RFID medium identifier verification result, account status verification result, and borrowing permissions; where the RFID medium identifier comes from the reading result of the RFID reader, the facial feature vector comes from the aligned facial image output by the facial feature extraction model, the account status information and borrowing constraint parameters come from the reader account records in the library's reader database, and the cosine similarity and available borrowing space are calculated by the formula given in this step.

[0107] Step 3: Borrowing and returning execution control and status confirmation;

[0108] In step three, the self-service borrowing terminal based on the transparent touch screen performs a physical book delivery action that matches the borrowing or returning of books according to the operation intention and borrowing permission in the interaction session parameters, and confirms the key status during the action execution process, and writes the status variables and confirmation results in the interaction session parameters.

[0109] The self-service borrowing terminal is pre-configured with a book-retrieval mechanism, an entrance door lock actuator, and a set of status sensors. The book-retrieval mechanism is defined in this design as an execution device capable of retrieving and placing books inside the cabinet, comprising at least a three-degree-of-freedom motion mechanism and a gripper; the three-degree-of-freedom motion mechanism enables displacement along three independent directions, and the gripper grips and releases the books. The entrance door lock actuator controls the lock status of the book outlet and the book return entrance. The set of status sensors includes at least: a photoelectric sensor located at the book outlet or book return entrance, a position feedback sensor located on the book-retrieval mechanism, and a door lock status signal acquisition device located at the entrance door lock. The position feedback sensor can be an encoder or a linear displacement sensor, and the door lock status signal is a discrete state quantity indicating whether the door is open or closed.

[0110] I. Book borrowing execution control when the operation intent is to borrow books;

[0111] When the operation intent in the interactive session parameters is to borrow books, the self-service borrowing terminal first reads the borrowing permissions in the interactive session parameters, and then reads the remaining borrowable amount R in the borrowing permissions. The self-service borrowing terminal determines whether R is greater than zero; when R≤0, the self-service borrowing terminal does not proceed with the book retrieval action, and directly writes the state of "book retrieval not started" into the interactive session parameters; when R>0, the self-service borrowing terminal proceeds with the book retrieval action.

[0112] After initiating the book retrieval action, the self-service borrowing terminal identifies the target book. In this solution, the self-service borrowing terminal uses a method of prioritizing target shelf identifier location and confirming the target book identifier with an RFID tag to determine the target book: First, it reads the target shelf identifier from the interactive session parameters and then looks up the target shelf's position coordinates in a table based on the identifier. The target shelf position coordinates are defined in this solution as coordinate data indicating the spatial position of the target shelf in the cabinet coordinate system, derived from a shelf position coordinate mapping table established during the equipment deployment phase. This shelf position coordinate mapping table is defined as a configuration table indexed by shelf area numbers and using the spatial coordinates of the corresponding shelf as values, obtained through installation, debugging, and measurement, and then permanently stored. Subsequently, the self-service borrowing terminal controls an RFID reader to perform RFID reading on the reading area corresponding to the target shelf, obtaining at least one book RFID tag. The book RFID tag is defined in this solution as a unique number of an RFID tag attached to or embedded in a physical book, derived from the reading result of the RFID reader. The self-service borrowing terminal matches the RFID tag identifier of the book it reads with the on-shelf record in the local collection index or the cloud collection index to obtain the RFID tag identifier of the target book. When there are multiple RFID tags, the self-service borrowing terminal determines the RFID tag identifier of the target book according to the unique result of the successful match and writes it into the interactive session parameters.

[0113] After the target shelf's position coordinates are determined, the self-service borrowing terminal controls the book-retrieval mechanism to complete continuous actions of movement, gripping, transfer, and delivery. To ensure the three-degree-of-freedom motion mechanism reaches the target shelf's position coordinates, the self-service borrowing terminal calculates the position error based on the current position feedback signal and the target shelf's position coordinates, and uses a proportional-integral-derivative (PID) control algorithm to generate the drive control quantity. This PID control algorithm satisfies the following:

[0114] ;

[0115] Where u(t) represents the drive control quantity at time t, and e(t) represents the position error at time t. Represents the proportionality coefficient. Represents the integral coefficient. The coefficient represents the differential coefficient; the position error e(t) is determined by the difference between the current position output by the position feedback sensor and the coordinates of the target book shelf position. The coefficient is a device control parameter and is set during deployment and debugging. The self-service borrowing terminal applies the drive control quantity to the corresponding drive motors of the three-degree-of-freedom motion mechanism, causing the gripper to move to the target book shelf position coordinates, and determines the positioning status based on the feedback signal from the position feedback sensor; when the positioning status is in place, the self-service borrowing terminal controls the gripper to perform a clamping action, thereby clamping the physical book in the target book shelf.

[0116] After clamping, the self-service borrowing terminal controls the three-degree-of-freedom motion mechanism to move along a preset transport path to the book outlet position coordinates. The book outlet position coordinates are defined in this scheme as: the spatial position coordinates of the book outlet in the cabinet coordinate system, derived from the book outlet position coordinate configuration fixed during the equipment deployment phase. After the book retrieval execution mechanism reaches the book outlet position coordinates, the self-service borrowing terminal controls the entrance door lock actuator to switch the book outlet door lock to the open state and reads the door lock status signal to confirm that the book outlet door lock is in the open state. When the book outlet door lock is in the open state, the self-service borrowing terminal controls the clamp to perform a release action, allowing the physical book to be delivered to the book outlet position, and the photoelectric sensor located at the book outlet detects the book's arrival status. After detecting the book's arrival status, the self-service borrowing terminal writes the book's arrival status to the interactive session parameters and continuously reads the photoelectric sensor signal to detect the book's departure status. When the photoelectric sensor changes from obstructed to unobstructed and continuously meets the preset holding time, the self-service borrowing terminal determines that the book has been taken and writes the confirmation result of the book's removal to the interactive session parameters. After the book is taken and confirmed, the self-service borrowing terminal controls the entrance door lock actuator to switch the book outlet door lock to the closed state and reads the door lock status signal to confirm that the book outlet door lock is in the closed state.

[0117] During the book borrowing process, the self-service borrowing terminal also performs a verification read of the target book's RFID tag at the book outlet. Specifically, the self-service borrowing terminal uses an RFID reader / writer located near the book outlet to read the RFID tags of books in the outlet area and compares the reading result with the target book's RFID tag in the interactive session parameters. If the comparison passes, the self-service borrowing terminal writes the result to the interactive session parameters; if the comparison fails, the self-service borrowing terminal writes the result to the interactive session parameters and switches the book retrieval mechanism to a retrieval action, causing the gripper to return the physical book to the preset retrieval position coordinates. The preset retrieval position coordinates are defined in this solution as: spatial coordinates used to temporarily store abnormal books, derived from the retrieval position coordinate configuration fixed during the equipment deployment phase.

[0118] II. Book return execution control when the operation intention is to return a book;

[0119] When the operation intent in the interaction session parameters is to return a book, the self-service borrowing terminal reads the borrowing permission in the interaction session parameters and determines whether the borrowing permission allows the book to be returned. When the borrowing permission is denied, the self-service borrowing terminal does not proceed with the book placement action, but directly writes the state of "returning the book has not been started" into the interaction session parameters. When the borrowing permission is allowed to be returned, the self-service borrowing terminal proceeds with the book placement action.

[0120] After the book placement action is initiated, the self-service borrowing terminal reads the book return entrance readiness status from the interactive session parameters and reads the door lock status signal to confirm the book return entrance door lock status. When the book return entrance readiness status is "ready" and the book return entrance door lock status is "open," the self-service borrowing terminal allows the physical book to enter the book return entrance, and the photoelectric sensor located at the book return entrance detects the book's arrival status. After detecting the book's arrival status, the self-service borrowing terminal controls the RFID reader to read the book's RFID tag in the book return entrance area and writes the read book's RFID tag identifier into the interactive session parameters. To avoid the occurrence of book numbers of unknown origin, the self-service borrowing terminal uses this book's RFID tag identifier as the unique identifier for this return, and does not replace the RFID reader's reading result with touch selection or manual input.

[0121] After the return object is determined, the self-service borrowing terminal determines the storage location coordinates and controls the book retrieval mechanism to perform the storage action. The storage location coordinates are defined in this scheme as: spatial coordinate data indicating the location where the returned book should be placed, derived from the joint determination result of the book shelf location coordinate mapping table and the inventory strategy; the inventory strategy is defined in this scheme as: a set of preset rules for selecting the target book shelf area number, including at least the original shelf priority rule and the empty shelf priority rule. Specifically, the self-service borrowing terminal queries the library index based on the book's RFID tag to obtain the original shelf area number of the book; when the shelf corresponding to the original shelf area number is in an empty and available state, the self-service borrowing terminal uses the original shelf area number as the target shelf identifier and obtains the storage location coordinates from the book shelf location coordinate mapping table; when the shelf corresponding to the original shelf area number is in an unavailable state, the self-service borrowing terminal selects an empty shelf area number from the preset candidate shelf set as the target shelf identifier and obtains the storage location coordinates from the book shelf location coordinate mapping table. The preset candidate book grid set is defined in this scheme as: a set of book grid area numbers that satisfy size matching and classification constraints, which is obtained by device deployment and configuration.

[0122] It should be noted that the preset candidate book grid set is used as the set of book grid area numbers for selecting storage locations based on inventory strategies in the book return scenario; the preset candidate book grid sequence set is used for stability determination in the book borrowing or book query scenario. The two are applicable to different business branches and are not used interchangeably.

[0123] The self-service borrowing terminal controls the book retrieval mechanism to move from the book return entrance coordinates to the book placement coordinates and perform a gripping action. Then, using the same proportional-integral-derivative (PID) control algorithm as the borrowing action, it controls the three-degree-of-freedom motion mechanism to move to the storage position coordinates. After the book retrieval mechanism reaches the storage position coordinates, the self-service borrowing terminal controls the gripper to release the physical book and confirms the placement completion status through a position feedback sensor and a status sensor at the book compartment entrance. After placement, the self-service borrowing terminal controls the entrance door lock actuator to switch the book return entrance door lock to the closed state, reads the door lock status signal to confirm that the book return entrance door lock is in the closed state, and simultaneously writes the confirmation result of the book being stored into the interactive session parameters.

[0124] III. Execution control when the operation intent is e-book reading, book search, or voice help;

[0125] When the intended operation is e-book reading, the self-service borrowing terminal reads the unique identifier of the target e-book from the interactive session parameters and loads the e-book content corresponding to the unique identifier on the transparent touch screen. The e-book content comes from the e-book resource index table preset in the terminal or the resource address pointed to by the result returned by the cloud resource directory. The self-service borrowing terminal writes the status of the loaded e-book content into the interactive session parameters.

[0126] When the operation intent is to search for books, the self-service borrowing terminal reads the touch context identifier or voice context text in the interaction session parameters, performs a search in the collection index, and obtains a search result list; the search result list comes from the matching output of the collection index, and the self-service borrowing terminal writes the generated search result list status into the interaction session parameters.

[0127] When the intended action is voice assistance, the self-service borrowing terminal reads the voice context text from the interaction session parameters and performs semantic matching in the local knowledge base or cloud-based question-and-answer interface to obtain the response text. The self-service borrowing terminal then writes the generated response text into the interaction session parameters. The response text is the output of the semantic matching and does not include any monitoring data collection on the user's identity or behavior.

[0128] Step 4: Result writing and multi-channel feedback presentation;

[0129] In step four, the self-service borrowing terminal based on the transparent touch screen writes the borrowing and returning execution results obtained in step three, and outputs the status after writing the results in the form of transparent touch screen presentation, voice broadcast presentation, or a combination of both; the result writing is limited to the data writing necessary for the book borrowing and returning business, and does not introduce user monitoring data collection that is unrelated to the business.

[0130] The self-service borrowing terminal is pre-configured with a library business server interface, a collection index, and a local transaction log. The library business server interface is defined in this solution as a communication interface for data exchange with the library business server, using any one of wired Ethernet, wireless LAN, or cellular mobile communication networks. The library business server is defined in this solution as a server for storing reader account records and collection borrowing / returning records. The collection index is defined in this solution as a data structure for mapping book RFID tags to collection records and providing retrieval capabilities; its source is either synchronization with the library business server or pre-configured locally on the terminal. The local transaction log is defined in this solution as sequentially stored data recording key write events and write statuses within the current interaction session, used for reconciliation and recovery in case of communication failures.

[0131] When the operation intent in the interaction session parameters is to borrow a book and the confirmation result written in step three includes that the book has been taken, the self-service borrowing terminal initiates a borrowing write using the RFID media identifier and the target book's RFID tag identifier in the interaction session parameters as the business primary key. The self-service borrowing terminal constructs a borrowing write request and sends it to the library's business server interface. The borrowing write request includes at least: the RFID media identifier, the target book's RFID tag identifier, the borrowing timestamp, and the interaction session identifier; wherein the borrowing timestamp originates from the terminal's local clock or the time synchronization result with the library's business server, and the interaction session identifier originates from the interaction session identifier generated in step one. After receiving the borrowing write request, the library's business server returns a write response, which includes at least a write status code and a borrowing record identifier; wherein the write status code indicates whether the write was successful or failed, and the borrowing record identifier is generated and returned by the library's business server. After receiving the write response, the self-service borrowing terminal writes the write status code and the borrowing record identifier into the interaction session parameters and simultaneously writes them into the local transaction log to form a traceable record.

[0132] When the write status code indicates a successful write, the self-service borrowing terminal updates the status of the library book record in its local collection index, switching the "on-shelf" status of the book record corresponding to the target book's RFID tag to the "checked out" status. The "on-shelf" status is derived from the library book record field and is not inferred by the terminal. The self-service borrowing terminal also simultaneously updates the borrowed item count field in the reader's account record: the terminal reads the current borrowed item count based on the reader's account record returned by the library's business server, increments the current borrowed item count by one based on the successful write operation, and writes the updated current borrowed item count into the interaction session parameters and records it in the local transaction log.

[0133] After completing the result writing and local status update, the self-service borrowing terminal performs multi-channel feedback presentation. The transparent touchscreen presentation includes at least: displaying a successful borrowing notification, displaying the library's book collection record information for the target book, and displaying the updated current number of borrowed books and the remaining borrowable quantity R. The formula for calculating the remaining borrowable quantity R has been given in step two; step four only reads R from the interactive session parameters or based on the data written in step two. and R is recalculated without introducing a new calculation method. The voice broadcast is executed by the speech synthesis engine: the self-service borrowing terminal inputs the successful borrowing notification text into the speech synthesis engine, obtains the voice broadcast signal, and outputs it through a speaker; the speech synthesis engine is defined in this scheme as: an algorithmic component that converts text into speech waveforms, and its output is playable speech waveform data.

[0134] When the write status code indicates a write failure, the self-service borrowing terminal writes the write failure status to the interaction session parameters and to the local transaction log. At the same time, it displays a write failure prompt on the transparent touch screen and broadcasts the write failure prompt text through a speech synthesis engine. The write failure prompt text contains at least a failure reason code or retry guidance information. The failure reason code comes from the write status code or interface error code returned by the library business server.

[0135] When the operation intent in the interaction session parameters is to return a book and the confirmation result written in step three includes that the book has been stored, the self-service borrowing terminal initiates a return write using the RFID medium identifier in the interaction session parameters and the RFID tag identifier of the book read from the return entry as the business primary key. The self-service borrowing terminal constructs a return write request and sends it to the library business server interface. The return write request includes at least: the RFID medium identifier, the RFID tag identifier of the book, the return timestamp, and the interaction session identifier; wherein the return timestamp comes from the terminal's local clock or the time synchronization result with the library business server, and the interaction session identifier comes from the interaction session identifier generated in step one. After receiving the return write request, the library business server returns a write response, which includes at least a write status code and a return record identifier; wherein the return record identifier is generated and returned by the library business server. The self-service borrowing terminal writes the write status code and the return record identifier into the interaction session parameters and writes them into the local transaction log.

[0136] When the write status code indicates successful writing, the self-service borrowing terminal updates the status of the library's catalog entries, switching the "on-shelf" status of the book entry corresponding to the book's RFID tag to "on-shelf," and writes this information into the interaction session parameters. Simultaneously, the self-service borrowing terminal reads the reader account records returned by the library's business server, obtains the updated number of currently borrowed books, and writes this number into the interaction session parameters and the local transaction log. The self-service borrowing terminal displays a successful return notification on the transparent touchscreen, along with the library's catalog entry information for the returned book; the terminal also uses a speech synthesis engine to read the successful return notification aloud.

[0137] When a write status code indicates a write failure, the self-service borrowing terminal writes the write failure status to the interaction session parameters and to the local transaction log. At the same time, it presents the write failure prompt text through the transparent touch screen and the speech synthesis engine. The write failure prompt text includes a failure reason code, which comes from the write status code or interface error code returned by the library business server.

[0138] When the operation intent in the interactive session parameters is book search, the self-service borrowing terminal writes the search result list generated in step three into the interactive session parameters and displays the search result list on the transparent touchscreen. Each item in the search result list contains at least the library's book record information and its shelf status, which are derived from the library's index. While displaying the search result list, the self-service borrowing terminal supports touch selection based on the transparent touchscreen, writing the touch coordinates obtained from subsequent touches into the interactive session parameters, thus forming a continuous query session record. This continuous query session record only contains touch coordinates and timestamps, and does not include fields for monitoring and collecting user behavior.

[0139] When the operation intent in the interactive session parameters is e-book reading, the self-service borrowing terminal writes the e-book content loading status written in step three into the interactive session parameters and presents the e-book reading interface on the transparent touch screen. The e-book reading interface supports at least page-turning touch operation and zoom touch operation. The self-service borrowing terminal writes the page-turning touch event and zoom touch event output by the touch controller into the local transaction log as interactive session parameters for breakpoint recovery during the reading process. Breakpoint recovery is defined in this scheme as: when the interface is refreshed due to a short interruption within the same interactive session, the e-book reading position is restored to the position before the interruption.

[0140] When the operation intent in the interactive session parameters is voice assistance, the self-service borrowing terminal writes the response text generated in step three into the interactive session parameters and outputs it in the form of transparent touch screen text presentation and voice synthesis broadcast presentation; the response text comes from the semantic matching output results of the local knowledge base or cloud question and answer interface, and the response text does not contain repeated collection fields of reader identity.

[0141] Finally, after writing the result of the corresponding operation intent and presenting the multi-channel feedback, the self-service borrowing terminal executes the interaction session termination process. The self-service borrowing terminal stores the final version of the interaction session parameters in its local transaction log, and uploads the write event summary corresponding to the current interaction session identifier in the local transaction log to the library's business server interface for consistency reconciliation when the network connection is normal. In this scheme, consistency reconciliation is defined as a verification process that compares the borrowing record identifier or return record identifier on the library's business server side with the local transaction log record on the terminal side. The verification fields are derived from the results returned by the library's business server and the terminal's local transaction log record; no new fields are introduced. After the interaction session ends, the self-service borrowing terminal restores the transparent touchscreen to the main interface or standby interface and clears the temporary fusion feature vector used for intent determination within this interaction session. A temporary voice waveform cache is used to retain only the write results necessary for the borrowing and returning business and the local transaction log records.

[0142] This invention also proposes an interactive control system for a self-service borrowing terminal based on a transparent touchscreen, such as... Figure 2 As shown, it includes: a session intent mapping module, an identity verification and authorization module, a borrowing and returning execution confirmation module, a result writing and feedback module, and signal connections between the modules;

[0143] The session intent mapping module is used by the self-service borrowing terminal to obtain touch input and voice input within the interactive session and generate an interactive session identifier to determine the operation intent; when the operation intent is to borrow books or search for books, the screen touch coordinates are mapped to bookshelf plane coordinates, and the mapping is corrected according to the touch event sequence, voice context text and intent discrimination results.

[0144] The identity verification and authorization module is used by the self-service borrowing terminal to collect RFID media identifiers and facial images, complete identity verification, and generate borrowing permissions and write them into the interactive session parameters.

[0145] The borrowing and returning execution confirmation module is used by the self-service borrowing terminal to control the book retrieval execution mechanism to execute the borrowing or returning of books according to the operation intention and borrowing permission, and write the status confirmation result.

[0146] The results writing feedback module is used to write the borrowing and returning results to the library's business server interface and to the local transaction log, and present them on a transparent touch screen or through voice broadcast.

[0147] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0148] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0149] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and inventive constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0150] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0151] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0152] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A self-service borrowing terminal interactive control method based on a transparent touch screen, characterized in that, include: After a touch event or voice wake-up is triggered, the self-service borrowing terminal generates an interactive session identifier and writes the touch context identifier or voice context text into the interactive session parameters. Then, it determines the operation intention based on a multi-class discrimination model and writes it into the interactive session parameters. When the operation intent is to borrow books or search for books, the self-service borrowing terminal saves the initial calibration data based on at least four sets of calibration points collected during the installation and debugging phase and obtains the initial homography matrix as the initial mapping relationship. In the interactive session, it maintains the session-state homography matrix used to map the screen touch coordinates to the bookshelf plane coordinates and updates it recursively. Within a session, the terminal constructs touch point distribution parameters from the touch event sequence to obtain touch dispersion and touch center migration, constructs correction indication from the voice context text, and constructs intent uncertainty from the intent category probability distribution. Then, based on touch dispersion, migration, correction indication, and intent uncertainty, it calculates update coefficients to adjust the update step size and update the session homography matrix. The terminal performs spatial matching between the bookshelf plane coordinates obtained by mapping the current touch coordinates and the boundaries of the preset book area to obtain the candidate book area number. When the candidate book area number belongs to the preset candidate book sequence set and the touch dispersion meets the stability judgment threshold, the target book identifier is determined and solidified and written into the interaction session parameters. When the correction indication meets the preset conditions and the migration exceeds the threshold, the terminal presents at least four guide touch points on the transparent touch screen and collects the corresponding point pairs to update the session homography matrix. The self-service borrowing terminal collects RFID media identifiers and facial images, completes identity verification, and generates borrowing permissions, which are then written into the interactive session parameters. The self-service borrowing terminal controls the book retrieval mechanism to execute the borrowing or returning of books based on the user's intention and borrowing permissions, and writes the status confirmation result. The borrowing and returning results are written to the library's business server interface and the local transaction log, and presented via a transparent touchscreen or voice broadcast.

2. The interactive control method for a self-service borrowing terminal based on a transparent touchscreen according to claim 1, characterized in that: The self-service borrowing terminal displays the main interface on a transparent touch screen and simultaneously activates the touch controller and array microphone as input sources. When a touch event is detected or a voice segment that meets the preset wake-up conditions is detected, an interaction session identifier is generated to identify the current interaction session and the start time of the interaction session is recorded. Within the same interactive session, the self-service borrowing terminal reads the touch context identifier corresponding to the touch event and records the touch context identifier as the touch context identifier. It then performs speech recognition on the voice signal to obtain the voice context text.

3. The interactive control method for a self-service borrowing terminal based on a transparent touchscreen according to claim 2, characterized in that: The self-service borrowing terminal generates fusion features for intent discrimination based on the voice context text and the touch context identifier, and outputs the probability distribution of intent categories through a multi-class discrimination model to determine the operation intent. The operation intent is written into the interaction session parameters, and after the operation intent is determined, the matching target determination process is selected and executed according to the operation intent, so that the interaction session parameters are fixed when the interaction session ends.

4. The interactive control method for a self-service borrowing terminal based on a transparent touchscreen according to claim 3, characterized in that: Once the operation intent in the interactive session parameters is determined, the self-service borrowing terminal performs target determination and permission generation according to the operation intent: if it is to return a book, it switches the book return guidance interface, reads the book return entrance ready status given by the door lock status signal or the entrance status sensor signal and writes it into the interactive session parameters; if it is to read an e-book, it displays the e-book catalog, receives touch selection to determine the target e-book item and writes the unique identifier of the item into the interactive session parameters. If it is voice assistance, keep the voice interaction interface active and write the voice context text and its recognition timestamp into the interaction session parameters. Within the same interactive session, the self-service borrowing terminal activates the RFID reader and image acquisition device to obtain the RFID medium identifier and the reader's face image and writes them into the interactive session parameters. Subsequently, it performs face detection and key point alignment on the face image and generates a face feature vector through a face feature extraction model. Based on the RFID medium identifier, it retrieves the bound face feature vector and reader account status information from the library's reader database. By calculating the cosine similarity and comparing it with a preset similarity threshold, it obtains the face verification result. At the same time, it outputs the RFID medium identifier verification result and the account status verification result and writes them into the interactive session parameters. Finally, based on the operation intention and each verification result, it generates borrowing permissions and writes them into the interactive session parameters. When generating the allowed borrowing permissions, it reads the maximum number of books that can be borrowed and the number of books that have been borrowed from the library's reader database to calculate the remaining borrowing capacity and uses it as a constraint condition for the borrowing permissions.

5. The interactive control method for a self-service borrowing terminal based on a transparent touchscreen according to claim 4, characterized in that: The self-service borrowing terminal controls the borrowing and returning of books and writes the status confirmation result based on the operation intention and borrowing permission in the interactive session parameters: When the operation intention is to borrow books, the terminal first determines whether to start the borrowing process based on the remaining borrowing quantity. After starting, it reads the target book grid identifier to determine the location coordinates of the target book grid and obtains the target book's RFID tag identifier through RFID reading and matching, and writes it into the interactive session parameters. Then, it controls the book retrieval mechanism to reach the target position, pick up the book, and transfer it to the book outlet. It controls the entrance door lock actuator to unlock and sends out the book after the door lock status signal confirms the unlocking. The book outlet photoelectric sensor confirms the book's arrival and departure and writes the arrival status and taken-away confirmation results respectively. Subsequently, the RFID tag of the book read from the book outlet is compared with the RFID tag of the target book, and the comparison result is written into the interactive session parameters. If there is a discrepancy, the book is retrieved. When the operation intention is to return a book, the terminal first determines whether the borrowing permission allows the book to be returned. If the return is allowed, the book is allowed to enter the return entrance based on the readiness status of the return entrance and the door lock status signal. After confirming the location with a photoelectric sensor, the RFID tag of the book is read as the unique identifier of the returned object and written into the interactive session parameters. Then, the storage location coordinates are determined according to the book compartment location coordinate mapping table and the inventory strategy, and the book retrieval execution mechanism is controlled to complete the storage. The location feedback and the status of the book compartment entrance are used to confirm the placement is completed and the book storage confirmation result is written into the interactive session parameters, and the door lock of the return entrance is closed. When the operation intention is to read e-books, search for books, or provide voice assistance, the terminal performs content loading, collection retrieval, or semantic matching based on the unique identifier of the target e-book, the touch context identifier, or the voice context text in the interactive session parameters, and writes the corresponding result generation status into the interactive session parameters.

6. The interactive control method for a self-service borrowing terminal based on a transparent touchscreen according to claim 5, characterized in that: The self-service borrowing terminal performs the following write and feedback output based on the borrowing and returning confirmation results in the interactive session parameters: When the borrowing confirmation result is that the book has been taken, the terminal constructs a borrowing write request using the RFID medium identifier and the RFID tag identifier of the target book, sends it to the library business server interface, and receives the write response. It writes the write status code and borrowing record identifier into the interactive session parameters and records them in the local transaction log. If the write is successful, it updates the shelf status of the collection index and the current number of borrowed books. If the write fails, it writes the failure status and reason code. When the returning confirmation result is that the book has been put away, the terminal constructs a returning write request using the RFID medium identifier and the RFID tag identifier of the book, and writes the status code and returning record identifier. If the write is successful, it updates the shelf status of the collection index and writes the updated current number of borrowed books. If the write fails, it writes the failure status and reason code. The terminal then displays the write results on a transparent touchscreen or outputs them via voice synthesis. At the end of the interaction session, it writes the interaction session parameters to the local transaction log and uploads the write event summary for consistency reconciliation when the network is normal. At the same time, it clears the temporary data related to intent identification and retains only the write results and local transaction log records necessary for the borrowing and returning business.

7. A self-service borrowing terminal interactive control system based on a transparent touchscreen, used to implement the self-service borrowing terminal interactive control method based on a transparent touchscreen as described in any one of claims 1-6, characterized in that, include: The module includes a session intent mapping module, an identity verification and authorization module, a borrowing and returning execution confirmation module, a result writing and feedback module, and signal connections between the modules. The session intent mapping module is used to generate an interactive session identifier after a touch event or voice wake-up is triggered by the self-service borrowing terminal, and write the touch context identifier or voice context text into the interactive session parameters. Then, it determines the operation intent based on a multi-class discrimination model and writes it into the interactive session parameters. When the operation intent is to borrow books or search for books, the self-service borrowing terminal saves the initial calibration data based on at least four sets of calibration points collected during the installation and debugging phase and obtains the initial homography matrix as the initial mapping relationship. In the interactive session, it maintains the session-state homography matrix used to map the screen touch coordinates to the bookshelf plane coordinates and updates it recursively. Within a session, the terminal constructs touch point distribution parameters from the touch event sequence to obtain touch dispersion and touch center migration, constructs correction indication from the voice context text, and constructs intent uncertainty from the intent category probability distribution. Then, based on touch dispersion, migration, correction indication, and intent uncertainty, it calculates update coefficients to adjust the update step size and update the session homography matrix. The terminal performs spatial matching between the bookshelf plane coordinates obtained by mapping the current touch coordinates and the boundaries of the preset book area to obtain the candidate book area number. When the candidate book area number belongs to the preset candidate book sequence set and the touch dispersion meets the stability judgment threshold, the target book identifier is determined and solidified and written into the interaction session parameters. When the correction indication meets the preset conditions and the migration exceeds the threshold, the terminal presents at least four guide touch points on the transparent touch screen and collects the corresponding point pairs to update the session homography matrix. The identity verification and authorization module is used by the self-service borrowing terminal to collect RFID media identifiers and facial images, complete identity verification, and generate borrowing permissions and write them into the interactive session parameters. The borrowing and returning execution confirmation module is used by the self-service borrowing terminal to control the book retrieval execution mechanism to execute the borrowing or returning of books according to the operation intention and borrowing permission, and write the status confirmation result. The results are written to the feedback module, which writes the borrowing and returning results to the library business server interface and to the local transaction log, and presents them on a transparent touch screen or through voice broadcast.

Images