Intelligent interaction method and device for railway ticket business self-service and railway self-service terminal

Abstract

The invention provides an intelligent interaction method and device for railway ticket business self-service and a railway self-service terminal, and the method comprises the steps: responding to a mode selection operation of a user, and collecting the face image data of the interaction process of the user and the railway self-service terminal; recognizing an emotion recognition result of the user corresponding to the facial image data, and obtaining operation behavior data of the user; under the condition that the emotion recognition result and the operation behavior data meet the active service condition, the preset active guide strategy is executed to assist the user in completing ticket business operation, so that traditional passive operation is converted into an intelligent guide process in which the system can actively intervene, the problem that the interaction mode is single is effectively solved, the waiting time of the user is shortened, and the user experience is improved. The interaction experience of railway ticket business self-service is improved; through real-time synchronization of the mouth shape, the expression and the voice guide verbal skill of the virtual customer service image, anthropomorphic interaction experience is created, the operation psychological threshold of the user is reduced, and the one-stop service process is smoother.

Description

Technical Field

[0001] This invention relates to the field of railway ticketing self-service technology, and in particular to an intelligent interactive method, device and railway self-service terminal for railway ticketing self-service. Background Technology

[0002] Currently, railway ticketing self-service technology mainly focuses on automatic ticket vending machines, self-service ticket collection machines, and some terminal devices that integrate refund and change functions. Although these devices have improved user convenience through software and hardware optimization, they still have many limitations: for example, the refund process still relies on traditional OCR (Optical Character Recognition) recognition and manual input, requiring the insertion of physical tickets and ID cards and manual selection of train numbers, which is inefficient; in terms of functionality, most terminals separate ticketing, refund, and change functions, requiring users to switch between different devices or interfaces, making one-stop service impossible; the interaction process mostly uses single methods such as touch screens and voice prompts, lacking virtual customer service guidance.

[0003] Overall, existing technologies still have significant shortcomings in terms of process efficiency and function integration, which reduces the interactive experience of railway ticketing self-service. Summary of the Invention

[0004] This invention provides an intelligent interactive method, device, and railway self-service terminal for railway ticketing self-service, which addresses the significant shortcomings of existing technologies in terms of process efficiency and functional integration, and reduces the deficiencies in the interactive experience of railway ticketing self-service.

[0005] This invention provides an intelligent interactive method for railway ticketing self-service, comprising the following steps: In response to the user's mode selection operation, facial image data of the user during the interaction process with the railway self-service terminal is collected; Identify the user's emotion recognition result corresponding to the facial image data, and obtain the user's operational behavior data; If the emotion recognition result and the operation behavior data meet the conditions for proactive service, a preset proactive guidance strategy is executed to assist the user in completing the ticketing operation; The proactive guidance strategy includes displaying a virtual customer service avatar on the screen of the railway self-service terminal and simultaneously broadcasting voice guidance messages related to the progress of the ticketing operation; the lip movements and facial expressions of the virtual customer service avatar are synchronized with the content of the voice guidance messages in real time.

[0006] According to the intelligent interactive method for railway ticketing self-service provided by the present invention, the proactive service conditions include a first service condition and a second service condition. The first service condition is that the emotion recognition result is confused, and the operation behavior data shows that the user has not performed any effective operation within a preset time period; The second service condition is that the emotion recognition result is anxious, and the operation behavior data shows that the user repeatedly performs invalid operations within a short period of time.

[0007] According to the intelligent interactive method for railway ticketing self-service provided by the present invention, when the emotion recognition result is anxiety, the proactive guidance strategy further includes increasing the processing priority of the ticketing operation and / or simplifying the interactive animation of the display screen.

[0008] According to the present invention, a smart interactive method for railway ticketing self-service is provided, the method further comprising: Identify the target population category corresponding to the facial image data; When the target population category is a preset population category, voice data of the interaction process between the user and the railway self-service terminal is collected; The voice data is subjected to speech recognition to obtain recognized text, and the recognized text is subjected to intent understanding to obtain the ticketing requirements of the user, and the ticketing operation corresponding to the ticketing requirements is executed.

[0009] According to the present invention, a smart interactive method for railway ticketing self-service is provided, the method further comprising: When the ticketing operation is a refund operation, obtain the user identity information associated with the refund operation and the device identifier of the railway self-service terminal; The user identity information and the device identifier are sent to the risk control server for security verification, and the risk level returned by the risk control server is received. Based on the risk level and the facial image data, a liveness detection is performed on the user, and a ticketing operation is executed based on the liveness detection result.

[0010] According to the present invention, a smart interactive method for railway ticketing self-service is provided, the method further comprising: When a physical ticket is detected, optical character recognition is performed on the physical ticket to obtain ticket information; When the ticket information is verified, in response to the user's ticket recycling operation, a preset points reward is issued to the member account associated with the user's identity information.

[0011] The present invention also provides an intelligent interactive device for railway ticketing self-service, comprising the following units: The acquisition unit is used to acquire facial image data of the user during the interaction between the user and the railway self-service terminal in response to the user's mode selection operation. The acquisition unit is used to identify the emotion recognition result of the user corresponding to the facial image data, and to acquire the user's operation behavior data; The execution unit is used to execute a preset proactive guidance strategy to assist the user in completing the ticketing operation when the emotion recognition result and the operation behavior data meet the proactive service conditions. The proactive guidance strategy includes displaying a virtual customer service avatar on the screen of the railway self-service terminal and simultaneously broadcasting voice guidance messages related to the progress of the ticketing operation; the lip movements and facial expressions of the virtual customer service avatar are synchronized with the content of the voice guidance messages in real time.

[0012] The present invention also provides a railway self-service terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the intelligent interactive method for railway ticketing self-service as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the intelligent interactive method for railway ticketing self-service as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements an intelligent interactive method for railway ticketing self-service as described above.

[0015] The intelligent interactive method, device, and railway self-service terminal for railway ticketing self-service provided by this invention, on the one hand, executes a preset proactive guidance strategy to assist users in completing ticketing operations when the emotion recognition results and operational behavior data meet the conditions for proactive service. This transforms the traditional passive operation into an intelligent guidance process that the system can proactively intervene in, effectively solving the problems of single interaction methods and lack of guidance, reducing user confusion and waiting time due to unfamiliarity with the operation, and improving the interactive experience of railway ticketing self-service. On the other hand, by creating a human-like interactive experience through the real-time synchronization of the virtual customer service avatar's lip movements, facial expressions, and voice guidance, the psychological threshold for user operation is lowered, making the one-stop service process smoother. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is one of the flowcharts illustrating the intelligent interactive method for railway ticketing self-service provided by the present invention.

[0018] Figure 2 This is the second flowchart of the intelligent interactive method for railway ticketing self-service provided by the present invention.

[0019] Figure 3 This is a schematic diagram of the intelligent interactive device for railway ticketing self-service provided by the present invention.

[0020] Figure 4 This is a schematic diagram of the railway self-service terminal provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and that the objects distinguished by "first," "second," etc., are generally of the same class.

[0023] This invention provides an intelligent interactive method for railway ticketing self-service, which can be applied to railway self-service terminals and other devices. The method can be executed by the railway self-service terminal, or by specific functional modules or plug-ins deployed within the railway self-service terminal. Figure 1 This is one of the flowcharts illustrating the intelligent interactive method for railway ticketing self-service provided by the present invention, such as... Figure 1 As shown, the method includes steps 110, 120 and 130.

[0024] Step 110: In response to the user's mode selection operation, facial image data of the user during the interaction process with the railway self-service terminal is collected.

[0025] Specifically, in this embodiment, the railway self-service terminal can be a new-type automatic ticket vending machine (TVM) integrating a multimodal perception module and an artificial intelligence processing unit. The multimodal perception module may include a camera for collecting facial image data, such as a 3D structured light camera, and a microphone array for collecting voice data, etc., which are not specifically limited in this embodiment of the invention.

[0026] First, in response to a user's mode selection action, facial image data can be collected during the user's interaction with the railway self-service terminal. The mode selection action is the trigger condition for activating this intelligent interaction method. In one possible implementation, the mode selection action could be that the user actively clicks a button for intelligent guidance mode or a similar function on the interactive interface of the railway self-service terminal, thereby explicitly authorizing the system to collect their facial image data to provide intelligent services. In another possible implementation, the mode selection action could be an implicit activation enabled by default. For example, when the camera of the railway self-service terminal detects that a user has lingered in front of the terminal for a certain period of time and is looking at the screen, the system can automatically enter the intelligent interaction preparation state and begin collecting facial image data.

[0027] Here, the mode selection operation can include an AI customer service mode that requires AI customer service assistance and a self-service mode that does not require customer service assistance.

[0028] Facial image data refers to image or video stream data captured in real-time by cameras installed on railway self-service terminals, reflecting the user's facial features and expression changes. To more accurately identify the user's state, a 3D structured light camera is preferred, as it can acquire not only two-dimensional image information but also facial depth information, facilitating more precise expression recognition and liveness detection. The process of acquiring facial image data can involve continuous frame capture of the facial video stream.

[0029] Furthermore, after acquiring facial image data, preprocessing operations can be performed on the facial image data, including but not limited to normalization, sharpening, and denoising. Normalization refers to mapping each dimension of the data vector to a range between (0, 1) or (-1, 1), or mapping a certain norm of the data vector to 1.

[0030] Step 120: Identify the emotion recognition result of the user corresponding to the facial image data, and obtain the user's operation behavior data.

[0031] Specifically, after acquiring facial image data, the system can identify the user's emotion corresponding to the facial image data and obtain the user's operational behavior data. That is, after acquiring the user's facial image data, the AI ​​processing unit built into the railway self-service terminal will analyze the facial image data.

[0032] Here, the emotion recognition result is obtained by analyzing and processing facial image data to determine the user's current emotional state. Specifically, the collected facial image data can be input into a pre-trained emotion recognition model. The emotion recognition model can analyze changes in key facial points, such as eyebrows, corners of the eyes, and corners of the mouth, to output the user's current emotion category, such as happy, calm, confused, anxious, or angry. The emotion recognition result can be a label representing a specific emotion or a confidence score representing various emotions; this embodiment of the invention does not specifically limit this.

[0033] The emotion recognition model can be the UltraFace model, a cascaded multilayer convolutional neural network (CNN), a deep neural network (DNN), or a combination of CNN and DNN, etc. The embodiments of the present invention do not specifically limit this.

[0034] While recognizing user emotions, the system also needs to acquire user action data. This action data refers to data recorded by the system during user interaction with the railway self-service terminal's interface. Action data may include user clicks, swipes on the touchscreen, duration of time spent on specific pages, input content in input boxes, frequency of operations, and whether any valid operations were performed within a preset time period. This embodiment of the invention does not specifically limit this; the action data can be recorded and provided by the railway self-service terminal's operating system or application software.

[0035] Step 130: If the emotion recognition result and the operation behavior data meet the conditions for proactive service, execute a preset proactive guidance strategy to assist the user in completing the ticketing operation. The proactive guidance strategy includes displaying a virtual customer service avatar on the screen of the railway self-service terminal and simultaneously broadcasting voice guidance messages related to the progress of the ticketing operation; the lip movements and facial expressions of the virtual customer service avatar are synchronized with the content of the voice guidance messages in real time.

[0036] Specifically, after obtaining the user's emotion recognition results, if the emotion recognition results and operational behavior data meet the conditions for proactive service, a preset proactive guidance strategy can be executed to assist the user in completing the ticketing operation.

[0037] The proactive service condition is a set of preset rules or a decision model used to determine whether a user needs assistance. The proactive service condition is based on a comprehensive judgment of the user's emotion recognition results and operational behavior data. Proactive service conditions may include detecting that the user's emotion is negative and their operational behavior is stagnant; or detecting that the user's emotion is normal but their operational behavior shows obvious ineffective or repetitive patterns.

[0038] Once the conditions for proactive service are met, the system will execute a preset proactive guidance strategy. This strategy includes displaying a virtual customer service avatar on the screen of the railway self-service terminal and simultaneously broadcasting voice guidance messages related to the progress of the ticketing operation.

[0039] The virtual customer service avatar can be a 2D or 3D animated character. Its appearance can attract users' attention in a more approachable and human-like way, alleviating users' negative emotions. The virtual customer service avatar can be designed to resemble a railway employee to increase professionalism and trustworthiness.

[0040] The voice guidance script is a voice prompt related to the user's current ticketing operation progress. For example, if the system detects that the user has been lingering on the train selection page for an extended period without any action and recognizes a confused expression, the triggered proactive guidance strategy could be: a virtual customer service avatar appears in the corner of the screen and broadcasts the guidance script: "Hello, are you having trouble selecting a train? You can select your departure and arrival cities above and then click the search button." This kind of guidance related to the operation progress avoids providing broad and irrelevant help information, making the guidance more targeted and effective. Here, the voice guidance script can be played from a pre-recorded audio file or synthesized in real time using text-to-speech technology; this embodiment of the invention does not specifically limit this.

[0041] Here, ticketing operations may include ticket refunds, ticket changes, and ticket sales, etc., and this embodiment of the invention does not specifically limit these operations. It should be understood that the railway self-service terminal in this embodiment of the invention is a terminal that integrates functions such as ticket refunds, ticket changes, and ticket sales. When users have relevant ticketing needs, they can complete all operations on this railway self-service terminal without switching between different devices or operating interfaces, thus realizing a true one-stop service.

[0042] The virtual customer service avatar's lip movements and facial expressions are synchronized with the content of the voice guidance script in real time.

[0043] The method provided in this invention, on the one hand, executes a preset proactive guidance strategy to assist users in completing ticketing operations when the emotion recognition results and operational behavior data meet the conditions for proactive service. This transforms the traditional passive operation into an intelligent guidance process that the system can proactively intervene in, effectively solving the problems of single interaction methods and lack of guidance, reducing user confusion and waiting time due to unfamiliarity with the operation, and improving the interactive experience of railway ticketing self-service. On the other hand, by real-time synchronization of the virtual customer service avatar's lip movements, facial expressions, and voice guidance scripts, a humanized interactive experience is created, lowering the psychological threshold for user operation and making the one-stop service process smoother.

[0044] Based on the above embodiments, the proactive service conditions include a first service condition and a second service condition; The first service condition is that the emotion recognition result is confused, and the operation behavior data shows that the user has not performed any effective operation within a preset time period; The second service condition is that the emotion recognition result is anxious, and the operation behavior data shows that the user repeatedly performs invalid operations within a short period of time.

[0045] Specifically, the proactive service conditions include a first service condition and a second service condition. The first service condition is that the emotion recognition result is confusion, and the operation behavior data shows that the user has not performed any effective operation within a preset time period. This scenario typically corresponds to a user's state of confusion due to unfamiliarity with the process. For example, an elderly user who is not used to self-service devices may hesitate in front of a certain interface because there is too much information or they do not know where to click next. At this time, the emotion recognition model outputs a result of confusion. At the same time, the operation behavior monitoring module finds that the user has not made any effective screen clicks or input behavior for up to 30 seconds. When both conditions are met, the first service condition is constituted. The proactive guidance strategy triggered at this time can focus on introducing the functions of the current interface and guiding the next operation.

[0046] The second service condition is that the emotion recognition result is anxious, and the user's behavioral data shows that the user repeatedly performs invalid operations within a short period of time. This scenario typically corresponds to a user's anxious state due to operational obstacles or slow system response. For example, a user in a hurry might be unable to click the payment button because a certain prerequisite has not been met, such as not having selected a seat. They might repeatedly and rapidly click the grayed-out payment button within 10 seconds. At this time, the emotion recognition model identifies the anxious emotion from the user's furrowed brow and tense facial muscles. When both conditions are met simultaneously, the second service condition is constituted. The proactive guidance strategy triggered at this time can more directly point out the problem with its voice guidance script, such as: "Hello, please select a seat before you can make payment." The method provided in this invention includes two proactive service conditions: a first service condition and a second service condition. The first service condition is that the emotion recognition result is confusion, and the operation behavior data shows that the user has not performed any effective operation within a preset time period. The second service condition is that the emotion recognition result is anxiety, and the operation behavior data shows that the user repeatedly performs invalid operations within a short period of time. By dividing the proactive service conditions into a first service condition and a second service condition, this method can trigger more targeted guidance strategies, improve the accuracy and humanization of intelligent interaction, further improve the user experience, and enhance the intelligence level of railway ticketing self-service.

[0047] Based on the above embodiments, when the emotion recognition result is anxiety, the proactive guidance strategy further includes increasing the processing priority of the ticketing operation and / or simplifying the interactive animation of the display screen.

[0048] Specifically, when the emotion recognition result is anxiety, the proactive guidance strategy, in addition to displaying a virtual customer service avatar and broadcasting guiding messages, may also include increasing the processing priority of ticketing operations and / or simplifying the interactive animations on the display screen.

[0049] Here, prioritizing ticketing operations means that when the system detects a user's anxiety, it can dynamically allocate more system resources to the current user's session. For example, the scheduling priority of the user's task in the terminal operating system can be increased, allowing it to receive more CPU (Central Processing Unit) time; or when communicating with the backend railway ticketing system, its data packets can be marked as high priority to achieve a faster network response. This aims to speed up processing from a technical perspective, shorten user waiting time, and thus directly alleviate user anxiety.

[0050] Here, simplifying interactive animations on the display refers to the system temporarily disabling or shortening some unnecessary interface transition animations, loading animations, and other visual effects. For example, changing a smooth swipe animation when switching pages to an instantaneous switch, or replacing complex dynamic loading icons with a simple static text "Loading..." when loading data. While these animations can enhance the aesthetics of the interface under normal circumstances, any non-immediate response can exacerbate the frustration of impatient users. Simplifying animations can make the interface appear to respond faster, thus psychologically calming the user.

[0051] Based on the above embodiments, the method further includes: Step 210: Identify the target population category corresponding to the facial image data; Step 220: If the target population category is a preset population category, collect voice data of the interaction between the user and the railway self-service terminal. Step 230: Perform speech recognition on the speech data to obtain recognized text, perform intent understanding on the recognized text to obtain the ticketing requirements corresponding to the user, and execute the ticketing operation corresponding to the ticketing requirements.

[0052] Specifically, it can identify the target population category corresponding to facial image data. While collecting user facial image data, the system's built-in AI processing unit can also run a population classification model. The target population category is a pre-defined user group that may require special interaction methods. For example, the target population category could be foreigners or domestic residents.

[0053] When the target audience is a preset group, voice data is collected during the interaction between the user and the railway self-service terminal. In the case of a preset group of foreign nationals, the system can automatically activate the microphone array on the railway self-service terminal to prepare for receiving the user's voice commands.

[0054] Finally, the system performs speech recognition on the voice data to obtain the user's ticketing needs and executes the corresponding ticketing operations. When a user issues a voice command, the voice data collected by the microphone array is sent to the speech recognition engine, which converts it into recognized text. Next, the natural language understanding module performs intent understanding on the recognized text, extracting the user's ticketing needs. These needs include the type of ticketing operation and key information. The operation type includes purchasing, refunding, and querying tickets, while the key information includes destination, time, and quantity. Finally, based on the parsed ticketing needs, the system automatically fills in the corresponding information on the interactive interface or redirects to the appropriate function page to execute the ticketing operation. For example, if a user says, "I want a second-class seat to XX South tomorrow," the system will automatically fill in "XX South" as the destination, select "tomorrow" as the date, and filter for second-class trains.

[0055] In addition, if the system identifies a user as a foreigner through facial features or voice content, it can automatically pop up a multilingual selection menu and switch the voice recognition engine to the corresponding language to provide multilingual voice interaction services.

[0056] The method provided in this invention first identifies the user's group category and automatically activates the voice interaction process when the user is identified as belonging to a specific group. This achieves intelligent differentiation and personalized guidance of service modes, effectively solving the problem of traditional terminals having a single interaction method and being unable to adapt to the usage habits of different user groups. By accurately parsing user intentions through voice recognition and intent understanding and automatically executing corresponding ticketing operations, it provides an accessible interaction channel for special groups with operational difficulties, significantly lowering their usage threshold and reflecting the humanization and inclusivity of the service. Finally, by integrating visual recognition and voice interaction into a coherent intelligent service loop, it avoids the tedious process of users manually searching for functions in multiple interfaces, making it particularly suitable for highly integrated ticketing terminals, thereby improving the overall accuracy, ease of use, and user satisfaction of the service.

[0057] In related technologies, risk control systems often rely on backend rules to make judgments based on a single dimension such as ticket refund frequency, which makes it difficult to integrate multimodal data for comprehensive evaluation and is prone to misjudgment.

[0058] Based on the above embodiments, the method further includes: Step 31: If the ticketing operation is a refund operation, obtain the user identity information associated with the refund operation and the device identifier of the railway self-service terminal; Step 32: Send the user identity information and the device identifier to the risk control server for security verification, and receive the risk level returned by the risk control server; Step 33: Based on the risk level and the facial image data, perform liveness detection on the user, and execute ticketing operations based on the liveness detection results.

[0059] Specifically, when the ticketing operation is a refund operation, the system obtains the user's identity information associated with the refund operation and the device identifier of the railway self-service terminal. That is, when a user selects to process a refund at the self-service terminal, the system initiates this risk control process. First, the system requires the user to provide identification, such as swiping their ID card, thereby obtaining the user's identity information, such as their ID number. Simultaneously, the system automatically reads and records the device identifier of the current railway self-service terminal, such as the terminal's serial number, IP (Internet Protocol Address), or MAC (Media Access Control Address). This embodiment of the invention does not specifically limit this.

[0060] Then, the user identity information and device identifier can be sent to the risk control server for security verification, and the system will receive the risk level returned by the risk control server. For example, the system can package the obtained user identity information and device identifier and send them to a dedicated risk control server over the network.

[0061] Here, the risk control server incorporates a behavioral database and a risk rule engine to perform security checks on received business information. The checks include, but are not limited to: Rule 1, verifying the frequency of ticket refunds initiated by the same identity information (e.g., ID number) on the same device or across devices within a set time window. If the frequency exceeds a set threshold (e.g., 3 times), it is considered high-risk. Rule 2, verifying whether the same device identifier is associated with multiple different identity information for ticket refunds within a short period. If the frequency is abnormal, it is considered suspected scalping behavior and classified as high-risk. Rule 3, considering the business scenario, such as a short interval between ticket refund and departure time and a shortage of tickets for the corresponding train, the risk weight is appropriately increased. Based on a comprehensive evaluation of the above rules, the risk control server will return a classification conclusion such as low-risk, medium-risk, or high-risk.

[0062] Furthermore, based on risk level and facial image data, liveness detection can be performed on users, and ticketing operations can be executed based on the liveness detection results. The railway self-service terminal adopts corresponding processing strategies based on the received risk level: if the risk level is low, it directly enters the regular ticket refund process without additional verification; if the risk level is medium or high, a liveness detection process is forcibly initiated, requiring the user to cooperate with the camera to complete specified actions, such as blinking or shaking their head, and technologies such as 3D structured light cameras can be used to effectively prevent photos and videos from being leaked. The system will judge based on the liveness detection results: if verification is successful, the ticket refund operation continues; if it fails, the current request is rejected and the user is prompted to transfer to manual processing. For extremely high-risk situations such as triggering "scalper" rules, in addition to rejecting the operation, the system can also automatically lock the terminal's ticket refund function, requiring intervention from the backend administrator to unlock it.

[0063] Understandably, by establishing such a ticket refund verification mechanism based on user behavior profiles and real-time risk control rules, this embodiment can accurately identify and intercept suspicious ticket refund behaviors using self-service devices, especially cracking down on scalpers' malicious ticket hoarding and refund operations, ensuring the fairness and security of the ticketing system, while keeping the process smooth for users who are getting refunds normally, achieving a balance between security and convenience.

[0064] While some terminals in related technologies have paper ticket recycling functions, they lack incentive measures such as points rewards, resulting in low user participation.

[0065] Based on the above embodiments, the method further includes: When a physical ticket is detected, optical character recognition is performed on the physical ticket to obtain ticket information; When the ticket information is verified, in response to the user's ticket recycling operation, a preset points reward is issued to the member account associated with the user's identity information.

[0066] Specifically, upon detecting a physical ticket, optical character recognition (OCR) is performed on the ticket to obtain its information. This step is applicable when a user needs to return a paper ticket that has already been retrieved. The railway self-service terminal integrates a ticket recycling slot. When a user inserts a physical ticket into the slot, an internal sensor detects its presence. Subsequently, a scanning module within the slot scans the ticket and, using OCR technology, automatically reads and parses the text and QR code information on the ticket, thereby obtaining ticket information including train number, seat number, and identity information.

[0067] When the ticket information is verified, in response to the user's ticket recycling operation, a preset points reward is issued to the member account associated with the user's identity information. Specifically, the system compares the ticket information recognized by OCR with the data in the backend ticketing system to verify the ticket's authenticity and validity. Once verification is successful, the interface will display refund information and simultaneously show a ticket recycling option, such as a "Recycle Ticket, Earn Environmental Points" button.

[0068] If a user chooses to perform this ticket redemption operation, the system will execute the incentive distribution process simultaneously with the ticket refund. The system will locate the user's railway membership account based on the ticket or the identity information provided during login, and then call the membership system interface to distribute a preset points reward to that account. The reward can be a fixed number of points, such as 5 points, or other forms of incentives, such as a chance to win a prize. These points can later be used to redeem railway travel coupons or other gifts.

[0069] By combining the ticket refund process with ticket recycling and points incentives, not only is the environmentally friendly recycling of used tickets achieved, saving resources and reducing the station's waste disposal costs, but the points reward system also encourages users' environmentally friendly behavior, enhances user participation and satisfaction, and strengthens user loyalty to the railway membership system, achieving multiple benefits of environmental protection, value enhancement, and improved user experience.

[0070] Based on any of the above embodiments Figure 2 This is the second flowchart illustrating the intelligent interactive method for railway ticketing self-service provided by the present invention, as shown below. Figure 2 As shown, the method includes: First, the user selects a mode to determine if they need customer service assistance. If the user selects no, they enter self-service mode. The system guides the user to place their ID card (including passport, etc.), then uses OCR to recognize the ID card information and a 3D camera for liveness verification. If the liveness verification is successful, the system checks for unused tickets. If available, the system prompts the user to process changes or refunds, and the process ends. If the verification fails, a message indicating a mismatch between the person and the ID is displayed, and the process ends. If the user selects customer service assistance, they enter AI customer service mode. The system first performs facial recognition on the user, then determines if they are a foreign traveler. If so, it automatically switches to the English version; otherwise, it maintains the Chinese version. The system then receives voice data from a microphone, acquires images from the 3D camera, and the AI ​​processing unit performs intelligent processing to complete ticket purchase and change services, ending the process. This process achieves multimodal intelligent interaction based on user mode selection and identity verification. It ensures identity security through OCR and 3D camera combined with liveness detection, and meets different user needs through facial recognition and language selection, improving the intelligence and convenience of operation and effectively ensuring the security and efficiency of ticketing services.

[0071] The method provided in this invention can improve user experience: by using natural language processing and pre-filling of ID card information, the operation becomes extremely simple, shortening ticket purchase time and reducing learning costs; it can stimulate potential demand: through intelligent recommendations and itinerary planning, it can uncover users' potential travel intentions and increase the railway company's passenger revenue; it can achieve a service loop by unifying online and offline ticketing services on the TVM, reducing the pressure on manual ticket windows and achieving 24-hour unmanned service; in addition, this method can optimize operational efficiency by intelligently guiding passenger flow, reducing passenger queuing time, and improving the overall operational efficiency of the station and passenger satisfaction.

[0072] In this embodiment of the invention, an intelligent microphone and a 3D intelligent camera are used to accurately obtain the needs of railway passengers through an AI core processing unit, thereby improving the passenger travel experience. Through in-depth research, other services that require manual handling at passenger stations can be centralized into this intelligent device, realizing an integrated intelligent service device for passenger travel.

[0073] The intelligent interactive device for railway ticketing self-service provided by the present invention is described below. The intelligent interactive device for railway ticketing self-service described below and the intelligent interactive method for railway ticketing self-service described above can be referred to in correspondence.

[0074] Based on any of the above embodiments, the present invention provides an intelligent interactive device for railway ticketing self-service. Figure 3 This is a schematic diagram of the intelligent interactive device for railway ticketing self-service provided by the present invention, as shown below. Figure 3 As shown, the device includes: The acquisition unit 310 is used to acquire facial image data of the user during the interaction process between the user and the railway self-service terminal in response to the user's mode selection operation. The acquisition unit 320 is used to identify the emotion recognition result of the user corresponding to the facial image data, and to acquire the user's operation behavior data; The execution unit 330 is used to execute a preset proactive guidance strategy to assist the user in completing the ticketing operation when the emotion recognition result and the operation behavior data meet the proactive service conditions. The proactive guidance strategy includes displaying a virtual customer service avatar on the screen of the railway self-service terminal and simultaneously broadcasting voice guidance messages related to the progress of the ticketing operation; the lip movements and facial expressions of the virtual customer service avatar are synchronized with the content of the voice guidance messages in real time.

[0075] The device provided in this invention, on the one hand, executes a preset proactive guidance strategy to assist users in completing ticketing operations when the emotion recognition results and operational behavior data meet the conditions for proactive service. This transforms the traditional passive operation into an intelligent guidance process that the system can proactively intervene in, effectively solving the problems of single interaction methods and lack of guidance, reducing user confusion and waiting time due to unfamiliarity with the operation, and improving the interactive experience of railway ticketing self-service. On the other hand, by real-time synchronization of the virtual customer service avatar's lip movements, facial expressions, and voice guidance, a human-like interactive experience is created, lowering the psychological threshold for user operation and making the one-stop service process smoother.

[0076] Based on any of the above embodiments, the proactive service conditions include a first service condition and a second service condition; The first service condition is that the emotion recognition result is confused, and the operation behavior data shows that the user has not performed any effective operation within a preset time period; The second service condition is that the emotion recognition result is anxious, and the operation behavior data shows that the user repeatedly performs invalid operations within a short period of time.

[0077] Based on any of the above embodiments, when the emotion recognition result is anxiety, the proactive guidance strategy further includes increasing the processing priority of the ticketing operation and / or simplifying the interactive animation of the display screen.

[0078] Based on any of the above embodiments, a speech recognition unit is further included, wherein the speech recognition unit is specifically used for: Identify the target population category corresponding to the facial image data; When the target population category is a preset population category, voice data of the interaction process between the user and the railway self-service terminal is collected; The voice data is subjected to speech recognition to obtain recognized text, and the recognized text is subjected to intent understanding to obtain the ticketing requirements of the user, and the ticketing operation corresponding to the ticketing requirements is executed.

[0079] Based on any of the above embodiments, a liveness detection unit is further included, wherein the liveness detection unit is specifically used for: When the ticketing operation is a refund operation, obtain the user identity information associated with the refund operation and the device identifier of the railway self-service terminal; The user identity information and the device identifier are sent to the risk control server for security verification, and the risk level returned by the risk control server is received. Based on the risk level and the facial image data, a liveness detection is performed on the user, and a ticketing operation is executed based on the liveness detection result.

[0080] Based on any of the above embodiments, a recycling unit is further included, wherein the recycling unit is specifically used for: When a physical ticket is detected, optical character recognition is performed on the physical ticket to obtain ticket information; When the ticket information is verified, in response to the user's ticket recycling operation, a preset points reward is issued to the member account associated with the user's identity information.

[0081] Figure 4 This is a schematic diagram of the structure of the railway self-service terminal provided by the present invention, as shown below. Figure 4 As shown, the railway self-service terminal may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. The processor 410, communication interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute an intelligent interactive method for railway ticketing self-service. This method includes: in response to a user's mode selection operation, collecting facial image data of the user's interaction with the railway self-service terminal; identifying the user's emotion recognition result corresponding to the facial image data, and acquiring the user's operational behavior data; if the emotion recognition result and the operational behavior data meet the conditions for proactive service, executing a preset proactive guidance strategy to assist the user in completing the ticketing operation; the proactive guidance strategy includes displaying a virtual customer service avatar on the display screen of the railway self-service terminal and simultaneously broadcasting voice guidance scripts related to the progress of the ticketing operation; the lip movements and facial expressions of the virtual customer service avatar are synchronized in real time with the content of the voice guidance scripts.

[0082] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0083] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the intelligent interactive method for railway ticketing self-service provided by the above methods. The method includes: in response to a user's mode selection operation, collecting facial image data of the user's interaction with the railway self-service terminal; identifying the user's emotion recognition result corresponding to the facial image data, and obtaining the user's operation behavior data; and, when the emotion recognition result and the operation behavior data meet the conditions for proactive service, executing a preset proactive guidance strategy to assist the user in completing the ticketing operation. The proactive guidance strategy includes displaying a virtual customer service avatar on the display screen of the railway self-service terminal and simultaneously broadcasting voice guidance scripts related to the progress of the ticketing operation. The lip movements and facial expressions of the virtual customer service avatar are synchronized with the content of the voice guidance scripts in real time.

[0084] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements an intelligent interactive method for railway ticketing self-service provided by the methods described above. This method includes: in response to a user's mode selection operation, acquiring facial image data of the user interacting with a railway self-service terminal; identifying the user's emotion recognition result corresponding to the facial image data, and acquiring the user's operational behavior data; and, if the emotion recognition result and the operational behavior data meet the conditions for proactive service, executing a preset proactive guidance strategy to assist the user in completing the ticketing operation. The proactive guidance strategy includes displaying a virtual customer service avatar on the display screen of the railway self-service terminal and simultaneously broadcasting voice guidance scripts related to the progress of the ticketing operation; the lip movements and facial expressions of the virtual customer service avatar are synchronized in real time with the content of the voice guidance scripts.

[0085] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent interactive method for railway ticketing self-service, characterized in that, include: In response to the user's mode selection operation, facial image data of the user during the interaction process with the railway self-service terminal is collected; Identify the user's emotion recognition result corresponding to the facial image data, and obtain the user's operational behavior data; If the emotion recognition result and the operation behavior data meet the conditions for proactive service, a preset proactive guidance strategy is executed to assist the user in completing the ticketing operation; The proactive guidance strategy includes displaying a virtual customer service avatar on the screen of the railway self-service terminal and simultaneously broadcasting voice guidance messages related to the progress of the ticketing operation; the lip movements and facial expressions of the virtual customer service avatar are synchronized with the content of the voice guidance messages in real time.

2. The intelligent interactive method for railway ticketing self-service according to claim 1, characterized in that, The proactive service conditions include a first service condition and a second service condition; The first service condition is that the emotion recognition result is confused, and the operation behavior data shows that the user has not performed any effective operation within a preset time period; The second service condition is that the emotion recognition result is anxious, and the operation behavior data shows that the user repeatedly performs invalid operations within a short period of time.

3. The intelligent interactive method for railway ticketing self-service according to claim 2, characterized in that, When the emotion recognition result is anxiety, the proactive guidance strategy also includes increasing the processing priority of the ticketing operation and / or simplifying the interactive animation of the display screen.

4. The intelligent interactive method for railway ticketing self-service according to any one of claims 1 to 3, characterized in that, The method further includes: Identify the target population category corresponding to the facial image data; When the target population category is a preset population category, voice data of the interaction process between the user and the railway self-service terminal is collected; The voice data is subjected to speech recognition to obtain recognized text, and the recognized text is subjected to intent understanding to obtain the ticketing requirements of the user, and the ticketing operation corresponding to the ticketing requirements is executed.

5. The intelligent interactive method for railway ticketing self-service according to any one of claims 1 to 3, characterized in that, The method further includes: When the ticketing operation is a refund operation, obtain the user identity information associated with the refund operation and the device identifier of the railway self-service terminal; The user identity information and the device identifier are sent to the risk control server for security verification, and the risk level returned by the risk control server is received. Based on the risk level and the facial image data, a liveness detection is performed on the user, and a ticketing operation is executed based on the liveness detection result.

6. The intelligent interactive method for railway ticketing self-service according to any one of claims 1 to 3, characterized in that, The method further includes: When a physical ticket is detected, optical character recognition is performed on the physical ticket to obtain ticket information; When the ticket information is verified, in response to the user's ticket recycling operation, a preset points reward is issued to the member account associated with the user's identity information.

7. An intelligent interactive device for railway ticketing self-service, characterized in that, include: The acquisition unit is used to acquire facial image data of the user during the interaction between the user and the railway self-service terminal in response to the user's mode selection operation. The acquisition unit is used to identify the emotion recognition result of the user corresponding to the facial image data, and to acquire the user's operation behavior data; The execution unit is used to execute a preset proactive guidance strategy to assist the user in completing the ticketing operation when the emotion recognition result and the operation behavior data meet the proactive service conditions. The proactive guidance strategy includes displaying a virtual customer service avatar on the screen of the railway self-service terminal and simultaneously broadcasting voice guidance messages related to the progress of the ticketing operation; the lip movements and facial expressions of the virtual customer service avatar are synchronized with the content of the voice guidance messages in real time.

8. A railway self-service terminal, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the intelligent interactive method for railway ticketing self-service as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the intelligent interactive method for railway ticketing self-service as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the intelligent interactive method for railway ticketing self-service as described in any one of claims 1 to 6.

Images