Ticket selling method and ticket selling system based on voice recognition

Abstract

The present invention proposes a ticket selling method and a ticket selling system based on voice recognition, wherein a camera is used to detect whether there is a face image whose size, direction and stay time meet preset conditions in a shooting picture, the face image includes a mouth shape image, a voice signal input by a user is obtained through a microphone, a starting point and an end point of the voice signal input by the user are determined according to changes in the mouth shape image, denoising is performed on the voice signal, the user's voice data is extracted from the denoised voice signal, voice recognition is performed on the voice data to obtain text content corresponding to the voice data, and ticket information is generated and displayed according to the text content, thereby solving the problem of low success rate of voice recognition due to environmental noise in public places.

Description

Technical Field

[0001] The present invention relates to the technical field of self-service ticketing, and in particular to a ticketing method and a ticketing system based on voice recognition. Background Art

[0002] In order to ease the ticketing pressure of manual ticket windows, solve the problems of crowds and queues, and improve ticketing efficiency, a large number of public service places such as cinemas, subway stations, railway stations, and long-distance bus stations are equipped with a large number of self-service ticketing equipment for people to buy tickets by themselves. The advantage of using a self-service ticket machine to buy tickets is that you can use them immediately after purchase, and there will be no problem of refunding tickets due to temporary changes in itinerary. At the same time, you can also ask for advice and assistance from on-site staff at any time. However, the self-service ticket machine has the problem of difficult text input and needs to follow the guidance step by step, which greatly reduces its convenience. For some older or less literate groups, it is difficult to use a self-service ticket machine to buy tickets. In recent years, the self-service ticket machines with voice recognition function have solved this pain point. By inputting the key information required for ticket purchase, such as the time of entry or station, the movie you want to watch or the target station, you can quickly locate the movie tickets or tickets you need to buy. The operation is simple and shortens the time required to buy tickets. However, since self-service ticket machines are generally installed in public places with large flow of people, the environmental noise is very serious, which has a great impact on the voice recognition effect. Sometimes users are required to repeat loudly several times for successful recognition, which loses the advantage of high input efficiency. Summary of the invention

[0003] Based on the above problems, the present invention proposes a ticket selling method and a ticket selling system based on voice recognition, which solves the problem of low success rate of voice recognition caused by environmental noise in public places.

[0004] In view of this, a first aspect of the present invention proposes a ticket selling method based on voice recognition, comprising:

[0005] Detecting by a camera whether there is a face image in the captured image that meets preset conditions in terms of size, orientation and dwell time, wherein the face image includes a mouth shape image;

[0006] Acquire a voice signal input by a user through a microphone;

[0007] Determining the starting point and the end point of the voice signal input by the user according to the change of the mouth shape image;

[0008] performing denoising on the speech signal;

[0009] Extracting user voice data from the denoised voice signal;

[0010] Performing speech recognition on the sound data to obtain text content corresponding to the sound data;

[0011] The bill information is generated and displayed according to the text content.

[0012] Furthermore, in the above-mentioned ticket selling method based on voice recognition, before the step of acquiring the voice signal input by the user through the microphone, it also includes:

[0013] Continuously monitor environmental sound information through the microphone;

[0014] Analyzing whether there is periodic noise and echo noise corresponding to the periodic noise in the environmental sound information;

[0015] If the periodic noise exists, extracting the time characteristics and spectrum characteristics of the periodic noise;

[0016] If there is echo noise corresponding to the periodic noise, the time feature and the spectrum feature of the echo noise are extracted.

[0017] Furthermore, in the above-mentioned ticket selling method based on speech recognition, the step of performing denoising on the speech signal specifically includes:

[0018] If the periodic noise exists, generating a first inverted waveform corresponding to the periodic noise according to the time characteristics and frequency spectrum characteristics of the periodic noise;

[0019] If there is echo noise corresponding to the periodic noise, generating a second inverted waveform corresponding to the echo noise according to the time characteristics and spectrum characteristics of the echo noise;

[0020] The speech signal, the first inverted waveform and / or the second inverted waveform are input into an inverted noise reduction function to cancel the periodic noise and / or the echo noise.

[0021] Furthermore, in the above-mentioned ticket selling method based on voice recognition, the step of determining the starting point and the end point of the voice signal input by the user according to the change of the mouth shape image specifically includes:

[0022] Extracting a mouth shape image from each frame of a picture image having a face image taken by the camera;

[0023] Compare the mouth shape image in each frame of the screen image with the mouth shape image in the previous screen image with a face image at an interval of n frames, where n is a positive integer greater than or equal to 1;

[0024] Determine whether the user's mouth shape changes from a closed state to an open state;

[0025] The time point corresponding to the screen image when the user's mouth shape changes from a closed state to an open state for the first time is taken as the starting point of the voice signal;

[0026] Determine whether the user's mouth shape changes from an open state to a closed state and whether the closed state is maintained for a time exceeding a preset first threshold;

[0027] The end point of the voice signal is the time point when the user's mouth shape changes from an open state to a closed state for the last time.

[0028] Furthermore, in the above-mentioned ticket selling method based on speech recognition, the step of performing denoising on the speech signal further comprises:

[0029] Obtain the time point at which the user's mouth shape changes from a closed state to an open state and from an open state to a closed state each time, so as to obtain the time period corresponding to the open state and the closed state of the user's mouth shape;

[0030] The sound signal whose short-term energy exceeds a preset second threshold value during a time period when the user's mouth is in a closed state is removed from the speech signal.

[0031] Furthermore, in the above-mentioned ticket selling method based on speech recognition, the step of extracting the user's voice data from the denoised speech signal specifically includes:

[0032] The denoised speech signal is input into the clipping function to obtain the quasi-pitch period of the user's voice;

[0033] Inputting the quasi-pitch period and the speech signal into a sound waveform fitting function to fit the sound waveform of the user;

[0034] The user's voice data is extracted from the voice signal according to the fitting result.

[0035] Furthermore, in the above-mentioned ticket selling method based on speech recognition, the quasi-pitch period is composed of the pitch period value and its error range value.

[0036] Furthermore, in the above-mentioned ticket selling method based on voice recognition, the step of inputting the denoised voice signal into the clipping function to obtain the quasi-pitch period of the user's voice specifically includes:

[0037] determining a maximum peak value in a sound waveform of the user's sound data;

[0038] Taking the average of the maximum peak value and the average value of the peak value of the sound waveform during the time period when the user's mouth shape is in an open state as the clipping level;

[0039] Inputting the sound waveform and the clipping level into a clipping function to obtain a peak waveform;

[0040] Determine whether the peak waveform has a waveform that does not conform to the periodic rule in which the error value is not greater than a third threshold;

[0041] If the determination is yes, the clipping level is fine-tuned upwards and the step of inputting the sound waveform and the clipping level into a clipping function to obtain a peak waveform is continued;

[0042] When the determination is no, the pitch period value and its error range value are calculated based on the peak waveform.

[0043] Furthermore, in the above-mentioned ticket selling method based on voice recognition, the step of inputting the quasi-pitch period and the voice signal into a voice waveform fitting function to fit the voice waveform of the user specifically includes:

[0044] Inputting the pitch period value into an autocorrelation function;

[0045] Adjusting the coefficients of the autocorrelation function and fine-tuning the pitch period value within the error value range to generate a corresponding harmonic waveform;

[0046] Matching the harmonic waveform with the sound waveform of the speech signal;

[0047] The user's voice waveform is determined based on the matching result.

[0048] The second aspect of the present invention proposes a ticketing system, comprising a camera for acquiring a user's facial image and a mouth shape image, a microphone for monitoring ambient noise and user voice signals, a touch screen for receiving user touch input, a memory for storing computer programs, and a processor, wherein the processor is used to execute the computer program to implement any of the voice recognition-based ticketing methods described in the first aspect above.

[0049] The present invention proposes a ticket selling method and a ticket selling system based on voice recognition, wherein a camera is used to detect whether there is a face image whose size, direction and stay time meet preset conditions in a shooting picture, the face image includes a mouth shape image, a voice signal input by a user is obtained through a microphone, a starting point and an end point of the voice signal input by the user are determined according to changes in the mouth shape image, denoising is performed on the voice signal, the user's voice data is extracted from the denoised voice signal, voice recognition is performed on the voice data to obtain text content corresponding to the voice data, and ticket information is generated and displayed according to the text content, thereby solving the problem of low success rate of voice recognition due to environmental noise in public places. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a schematic flow chart of a ticket selling method based on voice recognition provided by an embodiment of the present invention;

[0051] Figure 2 is a schematic flow chart of a periodic noise reduction method provided by an embodiment of the present invention;

[0052] Figure 3 is a schematic flow chart of a method for determining a start point and an end point of a speech signal provided by an embodiment of the present invention;

[0053] Figure 4 is a schematic flow chart of a speech signal denoising method provided by an embodiment of the present invention;

[0054] Figure 5 is a schematic flow chart of a method for extracting user voice data provided by an embodiment of the present invention;

[0055] Figure 6 is a schematic flow chart of a pitch period calculation method provided by an embodiment of the present invention;

[0056] Figure 7 It is a schematic flow chart of a user voice waveform fitting method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0058] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0059] In the description of the present invention, the term "multiple" refers to two or more. Unless otherwise clearly defined, the orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. The terms "connection", "installation", "fixation", etc. should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", etc. can explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, the meaning of "multiple" is two or more.

[0060] In the description of this specification, the description of the terms "one embodiment", "some implementations", "specific embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0061] The technical solution of the present invention is applied to a self-service ticket machine, especially to a self-service ticket machine installed in a public place. The ticketing system running on the self-service ticket machine implements a ticketing method based on voice recognition proposed in the first aspect of the present invention, that is, the self-service ticket machine detects whether there is a face image whose size, direction and residence time meet the preset conditions in the shooting picture through a camera, the face image includes a mouth shape image, obtains the voice signal input by the user through a microphone, determines the starting point and end point of the voice signal input by the user according to the change of the mouth shape image, performs denoising on the voice signal, extracts the user's voice data from the denoised voice signal, performs voice recognition on the voice data to obtain the text content corresponding to the voice data, generates and displays ticket information according to the text content, and solves the problem of low success rate of voice recognition caused by environmental noise in public places. Specifically, the second aspect of the present invention proposes a ticketing system, including a camera for obtaining a user's face graphic and its mouth shape image, a microphone for monitoring environmental noise and user voice signals, a touch screen for receiving user touch input, a memory for storing computer programs and a processor, and the processor is used to execute the computer program to implement any of the ticketing methods based on voice recognition described in the first aspect above.

[0062] Refer to the following Figures 1 to 7 A ticket selling method and a ticket selling system based on voice recognition are described in accordance with some embodiments of the present invention.

[0063] like Figure 1 As shown, the first aspect of the present invention proposes a ticket selling method based on voice recognition, comprising:

[0064] S100: Detect by a camera whether there is a face image in the shooting picture whose size, direction and stay time meet the preset conditions, and the face image includes a mouth shape image. In the technical solution of the present invention, the face image of the user is detected by a camera set on the self-service ticket machine to determine whether there is a user who purchases tickets through the self-service ticket machine. Since the self-service ticket machine is installed in a public place where people come and go, a large number of face images of passers-by will appear in the shooting picture of the camera. It is necessary to judge whether the captured face image is a face image of a user with the intention of purchasing a ticket or a face image of a pedestrian passing by temporarily according to the size, direction and stay time of the face in the shooting picture. When the size, direction and stay time of the face in the shooting picture meet the preset conditions, for example, the proportion of the face image in the shooting picture is greater than 40%, the face is deflected by no more than 30 degrees in the direction of the self-service ticket machine, and the time that meets the above two conditions, that is, the stay time exceeds 5 seconds, it can be considered that the user corresponding to the face image is a user with the intention of purchasing a ticket.

[0065] S200: Acquire a voice signal input by the user through a microphone. In the technical solution of the present invention, a voice signal of inquiry or ticket purchase issued by the user is acquired through a microphone installed on the self-service ticket machine. While collecting the user's voice signal, the microphone also collects some environmental sounds such as the sound of the broadcast, the voice of passers-by, the sound of vehicles, and the sound of collision or friction when other objects move, etc. into the voice signal, so the voice signal also contains a large amount of noise signals.

[0066] S300: Determine the starting point and end point of the voice signal input by the user according to the change of the mouth shape image. When there is a face image that meets the above conditions in the shooting picture of the camera, extract the mouth shape image from the face image to determine the mouth shape change corresponding to the voice signal issued by the user according to the change of the mouth shape image, so that the starting time and end point time of the voice signal input by the user can be determined according to the time corresponding to the mouth shape change of the user.

[0067] S400: De-noising the speech signal. Remove obvious noise from the speech signal, such as some periodic noise and echo noise. In public places such as cinemas and subway stations, periodic noise is one of the main noise forms, such as periodic broadcast sounds or the sounds of subways entering and exiting the station. At the same time, since cinemas and subway stations are generally relatively closed places, echo noise is also one of the main noise forms.

[0068] S500: Extracting the user's voice data from the denoised voice signal. Since the user is in a position close to the self-service ticket machine when inputting the voice signal, that is, the position where the user makes the sound is close to the microphone on the self-service ticket machine, after removing the periodic noise and echo noise, there is no continuous sound information in the voice signal whose average short-time energy value is greater than the user's voice, that is, in the sound waveform of the denoised voice signal, most of the peak waveforms can be confirmed as the user's voice waveform, thereby making it possible to extract the user's voice data from the denoised voice signal. However, the denoised voice signal still contains other possible noises, such as the voices of passers-by talking, etc., and it is necessary to identify the user's voice data from the denoised voice signal and extract it to improve the accuracy of voice recognition. Specifically, the user's voice data is the user's voice waveform information extracted from the voice signal.

[0069] S600: Performing speech recognition on the sound data to obtain text content corresponding to the sound data. Inputting the extracted user's sound data into a pre-trained speech recognition model locally or in the cloud to recognize the user's sound data and convert it into corresponding text content.

[0070] S700: Generate and display ticket information according to the text content. For example, the text content recognized from the user's voice data is "query for tickets to XXX station within one hour" or "buy the earliest ticket to XXX", etc. The self-service ticket vending machine performs corresponding actions of generating and displaying ticket information according to the text content, and the user can perform subsequent payment and ticket printing operations according to the ticket information.

[0071] like Figure 2 As shown, in the above-mentioned ticket selling method based on voice recognition, before the step of acquiring the voice signal input by the user through the microphone, it also includes:

[0072] S110: Continuously monitoring environmental sound information through the microphone. In this embodiment, the microphone provided on the self-service ticket machine also continuously monitors environmental sound when no qualified user face image is detected.

[0073] S120: Analyze whether there is periodic noise and echo noise corresponding to the periodic noise in the environmental sound information. Based on the long-term analysis of environmental sound, the time characteristics and spectrum characteristics of the periodic noise and its corresponding echo noise can be found and extracted.

[0074] S130: If the periodic noise exists, extract the time feature and spectrum feature of the periodic noise. The time feature refers to the time when the periodic noise occurs and its time period, such as the sound of a broadcast that is played periodically to remind pedestrians to pay attention to safety, the time of its first broadcast every day and the time interval between its broadcasts, etc. The spectrum feature refers to the sound waveform corresponding to the periodic noise extracted by the ticketing system based on the comparison of sound waveforms extracted from multiple time periods.

[0075] S140: If there is echo noise corresponding to the periodic noise, extract the time characteristics and spectrum characteristics of the echo noise. When the self-service ticket machine is installed in an outdoor environment, the volume of the echo generated by the periodic noise indoors will also be very loud, and the echo corresponding to the periodic noise is also periodic, which is easier to identify and extract. Especially for some indoor environments with building sizes or internal furnishings that have not been specially designed for denoising, the periodic noise and its echo of multiple periods may even produce standing waves so that the amplitude of its sound waveform is greatly enhanced. When the self-service ticket machine is installed in a very open outdoor environment, the failure to generate an echo or the low volume of the echo covered by other sounds will not have an adverse effect on the recognition of the voice signal.

[0076] Continue to see Figure 2In the above-mentioned ticket selling method based on speech recognition, the step of performing denoising on the speech signal specifically includes:

[0077] S410: If the periodic noise exists, generate a first inverted waveform corresponding to the periodic noise according to the time characteristics and frequency spectrum characteristics of the periodic noise;

[0078] S420: If there is echo noise corresponding to the periodic noise, generate a second inverted waveform corresponding to the echo noise according to the time characteristics and frequency spectrum characteristics of the echo noise;

[0079] S430: Input the speech signal, the first inverted waveform and / or the second inverted waveform into an inverted noise reduction function to cancel the periodic noise and / or the echo noise.

[0080] In this embodiment, when there is periodic noise and its echo noise, a corresponding inverted waveform is generated according to its spectral characteristics. When the camera detects that a user is performing an inquiry or ticket purchase operation, the inverted waveform is superimposed on the signal of the corresponding time period of the voice signal in an inverted noise reduction function according to the time characteristics of the periodic noise and its echo noise to offset the periodic noise and its echo noise of the corresponding time period.

[0081] like Figure 3 As shown, in the above-mentioned ticket selling method based on voice recognition, the step of determining the starting point and the end point of the voice signal input by the user according to the change of the mouth shape image specifically includes:

[0082] S310: Extracting a mouth shape image from each frame of the image having a face image captured by the camera.

[0083] S320: Compare the mouth shape image in each frame of the screen image with the mouth shape image in the screen image with a face image at a previous interval of n frames, where n is a positive integer greater than or equal to 1. Preferably, the mouth shape image comparison is performed using a frame interval of n=m, where m is the sampling frame rate of the camera device, that is, the camera device uses a frame rate of m frames / second to collect images, and m is a positive integer. In this embodiment, after detecting a face image that meets the conditions, starting from the mth frame where the face image is detected, the mouth shape image of the user in each frame of the image captured by the camera is compared with the mouth shape image in the previous frame or another frame of the image at a preset interval, that is, the mouth shape image of each frame of the image is compared with the mouth shape image in the previous frame captured at an interval of 1 second. For a camera device with a higher preset frame rate, the time interval between two adjacent frames of the image is short. At the normal speaking speed of the user, the mouth shape changes slowly, and the mouth shape images of adjacent frames do not change significantly, so it is necessary to increase the frame interval between the compared images.

[0084] S330: Determine whether the user's mouth shape has changed from a closed state to an open state. By comparing the mouth shape images in the front and back shots, it is possible to determine whether the user's mouth shape has changed, and determine whether it has changed from a closed state to an open state, or from an open state to a closed state, based on the change state of the mouth shape, and determine the time corresponding to the change process. The user's mouth shape being in a closed state means that the user's upper and lower lips are in a completely closed state, and the user's mouth shape being in an open state means that the user's upper and lower lips are in a non-completely closed state.

[0085] S340: The time point corresponding to the screen image when the user's mouth shape changes from a closed state to an open state for the first time is used as the starting point of the voice signal. Starting from the time when a qualified face image is detected in the shooting screen, when the user's mouth shape changes from a closed state to an open state for the first time, the time point corresponding to the first frame of the shooting screen image when the user's mouth shape begins to show an open feature is used as the starting point of the voice signal.

[0086] S350: Determine whether the user's mouth shape changes from an open state to a closed state and whether the time in which the closed state is maintained exceeds a preset first threshold. Starting from the time point corresponding to the first time the user's mouth shape changes from a closed state to an open state, detect the situation where the user's mouth shape changes from an open state to a closed state. That is, after the user starts speaking, if the user's mouth shape is in a closed state for a time period exceeding the preset first threshold, for example, the user's mouth shape does not open again for more than 3 seconds or 5 seconds, it means that the user has stopped inputting the voice signal.

[0087] S360: The time point when the user's mouth shape last changes from an open state to a closed state is taken as the end point of the voice signal. When the user's mouth shape changes to a closed state and remains in the closed state for a duration greater than the preset first threshold, that is, when the user stops inputting the voice signal, the time point when the user's mouth shape last changes from an open state to a closed state before this is taken as the end point of the voice signal.

[0088] like Figure 4 As shown, in the above-mentioned ticket selling method based on voice recognition, the step of performing denoising on the voice signal further includes:

[0089] S410: Obtain the time point each time the user's mouth shape changes from a closed state to an open state and from an open state to a closed state, so as to obtain the time periods corresponding to the open and closed states of the user's mouth shape. The time of the user input voice signal is divided according to the user's mouth shape state, which can be divided into a time period when the mouth shape is in an open state and a time period when the mouth shape is in a closed state. The time of the user input voice signal is divided into a time period corresponding to the open state of the user's mouth shape and a time period corresponding to the closed state of the user's mouth shape according to the mouth shape image in the captured picture during the period when the user inputs the voice signal.

[0090] S420: Eliminate from the voice signal the sound signal whose short-time energy exceeds the preset second threshold value during the time period when the user's mouth shape is in a closed state. When the user is speaking, part of the pronunciation may be made when the user's mouth shape is closed, but it can be determined that the average short-time energy of the voice signal input in the time period corresponding to the user's open mouth shape is greater than the average short-time energy of the voice signal input in the time period corresponding to the user's closed mouth shape. When a sound signal whose average short-time energy exceeds the preset second threshold value appears in the time period when the user's mouth shape is closed, it can be determined that the sound signal is an abnormal signal or a noise signal, and the sound signal needs to be removed from the voice signal.

[0091] like Figure 5 As shown, in the above-mentioned ticket selling method based on voice recognition, the step of extracting the user's voice data from the denoised voice signal specifically includes:

[0092] S510: Input the denoised voice signal into the clipping function to obtain the quasi-pitch period of the user's voice. Furthermore, in the above-mentioned ticketing method based on voice recognition, the quasi-pitch period consists of the pitch period value and its error range value. The pitch period in the user's voice is not a strictly identical time period, and it may change within a certain error range. By inputting the denoised voice signal into the clipping function, the quasi-pitch period of the user in the voice signal can be obtained, and the quasi-pitch period consists of the pitch period value and its error range value.

[0093] S520: Input the quasi-pitch period and the speech signal into a sound waveform fitting function to fit the user's sound waveform.

[0094] S530: Extracting the user's voice data from the voice signal according to the fitting result.

[0095] like Figure 6 As shown, in the above-mentioned ticket selling method based on voice recognition, the step of inputting the denoised voice signal into the clipping function to obtain the quasi-pitch period of the user's voice specifically includes:

[0096] S511: Determine the maximum peak value in the sound waveform of the user's sound data;

[0097] S512: taking the average of the maximum peak value and the average value of the peak value of the sound waveform during the time period when the user's mouth is in an open state as the clipping level;

[0098] S513: Inputting the sound waveform and the clipping level into a clipping function to obtain a peak waveform;

[0099] S514: Determine whether the peak waveform has a waveform that does not conform to the periodic rule in which the error value is not greater than a third threshold;

[0100] S515: When the determination is yes, the clipping level is fine-tuned upwards and the step of inputting the sound waveform and the clipping level into a clipping function to obtain a peak waveform is continued;

[0101] S516: When the determination is no, the pitch period value and its error range value are calculated based on the peak waveform.

[0102] As mentioned above, in the step of performing denoising on the voice signal, the noise data with high short-time energy in the voice signal has been removed, and in the remaining sound data, the short-time energy value of the user's sound data is relatively high. It can be determined that most of the peak waveforms in the voice signal are sound waveforms in the user's sound data. When the peak waveform obtained by using a certain clipping level value still contains a waveform that does not conform to the periodic law within the error range, it means that the peak waveform still contains noise data, and the clipping level needs to be increased to regain the peak waveform.

[0103] like Figure 7 As shown, in the above-mentioned ticket selling method based on voice recognition, the step of inputting the quasi-pitch period and the voice signal into the voice waveform fitting function to fit the voice waveform of the user specifically includes:

[0104] S521: Input the pitch period value into the autocorrelation function;

[0105] S522: adjusting the coefficient of the autocorrelation function and fine-tuning the pitch period value within the error value range to generate a corresponding harmonic waveform;

[0106] S523: Matching the harmonic waveform with the sound waveform of the speech signal;

[0107] S524: Determine the user's voice waveform according to the matching result.

[0108] In the technical scheme of the above-mentioned implementation mode, the fundamental period value is dynamically adjusted within the error range, and the coefficients of different autocorrelation functions are adapted, so as to generate a corresponding harmonic waveform based on the fundamental waveform in the voice signal. When the matching degree of the waveform signals on both sides of the harmonic waveform and the corresponding fundamental waveform in the voice signal meets the preset conditions, for example, when the number of peaks whose offset value between the peak-to-peak time point in the harmonic waveform and the peak-to-peak time point at the corresponding position in the voice signal is not greater than the preset fourth threshold and is greater than the preset fifth threshold, the harmonic waveform is determined as the user's voice waveform, so that the user's voice data can be extracted from the voice signal based on the fundamental waveform and the harmonic waveform.

[0109] The second aspect of the present invention proposes a ticketing system, comprising a camera for acquiring a user's facial image and a mouth shape image, a microphone for monitoring ambient noise and user voice signals, a touch screen for receiving user touch input, a memory for storing computer programs, and a processor, wherein the processor is used to execute the computer program to implement any of the voice recognition-based ticketing methods described in the first aspect above.

[0110] The present invention proposes a ticket selling method and a ticket selling system based on voice recognition, wherein a camera is used to detect whether there is a face image whose size, direction and stay time meet preset conditions in a shooting picture, the face image includes a mouth shape image, a voice signal input by a user is obtained through a microphone, a starting point and an end point of the voice signal input by the user are determined according to changes in the mouth shape image, denoising is performed on the voice signal, the user's voice data is extracted from the denoised voice signal, voice recognition is performed on the voice data to obtain text content corresponding to the voice data, and ticket information is generated and displayed according to the text content, thereby solving the problem of low success rate of voice recognition due to environmental noise in public places.

[0111] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0112] According to the embodiments of the present invention as described above, these embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made based on the above description. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can make good use of the present invention and the modified use based on the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A ticket selling method based on voice recognition, characterized in that: include: Detecting by a camera whether there is a face image in the captured image that meets preset conditions in terms of size, orientation and dwell time, wherein the face image includes a mouth shape image; Acquire a voice signal input by a user through a microphone; Determining the starting point and the end point of the voice signal input by the user according to the change of the mouth shape image; performing denoising on the speech signal; Extracting user voice data from the denoised voice signal; Performing speech recognition on the sound data to obtain text content corresponding to the sound data; Generate and display bill information according to the text content; Before the step of acquiring the voice signal input by the user through the microphone, the method further includes: Continuously monitor environmental sound information through the microphone; Analyzing whether there is periodic noise and echo noise corresponding to the periodic noise in the environmental sound information; If the periodic noise exists, extracting the time characteristics and spectrum characteristics of the periodic noise; If there is echo noise corresponding to the periodic noise, extracting the time characteristics and spectrum characteristics of the echo noise; The step of determining the starting point and the end point of the voice signal input by the user according to the change of the mouth shape image specifically includes: Extracting a mouth shape image from each frame of a picture image having a face image taken by the camera; The mouth shape image in each frame of the screen image is compared with the mouth shape image in the screen image with a face image at a previous interval of n frames, and the mouth shape image comparison is performed at a frame interval of n=m, wherein m is the sampling frame rate of the camera, and the camera uses a frame rate of m frames / second for image acquisition, and m is a positive integer. After a face image that meets the conditions is detected, starting from the mth frame where the face image is detected, the mouth shape image of the user in each frame of the image taken by the camera is compared with the mouth shape image in the previous frame or another frame of the image at a preset interval; By comparing the mouth shape images in the front and back shots, it is determined whether the user's mouth shape has changed from a closed state to an open state. The closed state of the user's mouth shape means that the upper and lower lips of the user are in a completely closed state, and the open state of the user's mouth shape means that the upper and lower lips of the user are not completely closed. The time point corresponding to the screen image when the user's mouth shape changes from a closed state to an open state for the first time is taken as the starting point of the voice signal; Determine whether the user's mouth shape changes from an open state to a closed state and whether the closed state is maintained for a time exceeding a preset first threshold; The end point of the voice signal is the time point when the user's mouth shape changes from an open state to a closed state for the last time; The step of performing denoising on the speech signal also includes: Obtain the time point at which the user's mouth shape changes from a closed state to an open state and from an open state to a closed state each time, so as to obtain the time period corresponding to the open state and the closed state of the user's mouth shape; The sound signal whose short-term energy exceeds a preset second threshold value during a time period when the user's mouth is in a closed state is removed from the speech signal.

2. The ticket selling method based on voice recognition according to claim 1, characterized in that: The step of performing denoising on the speech signal specifically includes: If the periodic noise exists, generating a first inverted waveform corresponding to the periodic noise according to the time characteristics and frequency spectrum characteristics of the periodic noise; If there is echo noise corresponding to the periodic noise, generating a second inverted waveform corresponding to the echo noise according to the time characteristics and spectrum characteristics of the echo noise; The speech signal, the first inverted waveform and / or the second inverted waveform are input into an inverted noise reduction function to cancel the periodic noise and / or the echo noise.

3. The ticket selling method based on voice recognition according to claim 1 or 2, characterized in that: The steps of extracting the user's voice data from the denoised voice signal specifically include: The denoised speech signal is input into the clipping function to obtain the quasi-pitch period of the user's voice; Inputting the quasi-pitch period and the speech signal into a sound waveform fitting function to fit the sound waveform of the user; The user's voice data is extracted from the voice signal according to the fitting result.

4. The ticket selling method based on voice recognition according to claim 3, characterized in that: The quasi-pitch period consists of a pitch period value and an error range value thereof.

5. The ticket selling method based on voice recognition according to claim 4, characterized in that: The step of inputting the denoised speech signal into the clipping function to obtain the quasi-pitch period of the user's voice specifically includes: determining a maximum peak value in a sound waveform of the user's sound data; Taking the average of the maximum peak value and the average value of the peak value of the sound waveform during the time period when the user's mouth shape is in an open state as the clipping level; Inputting the sound waveform and the clipping level into a clipping function to obtain a peak waveform; Determine whether the peak waveform has a waveform that does not conform to the periodic rule in which the error value is not greater than a third threshold; If the determination is yes, the clipping level is fine-tuned upwards and the step of inputting the sound waveform and the clipping level into a clipping function to obtain a peak waveform is continued; When the determination is no, the pitch period value and its error range value are calculated based on the peak waveform.

6. The ticket selling method based on voice recognition according to claim 4 or 5, characterized in that: The step of inputting the quasi-pitch period and the voice signal into a sound waveform fitting function to fit the user's sound waveform specifically includes: Inputting the pitch period value into an autocorrelation function; Adjusting the coefficients of the autocorrelation function and fine-tuning the pitch period value within the error range value to generate a corresponding harmonic waveform; Matching the harmonic waveform with the sound waveform of the speech signal; The user's voice waveform is determined based on the matching result.

7. A ticketing system, characterized in that: The invention comprises a camera for acquiring a user's face pattern and a mouth shape image thereof, a microphone for monitoring environmental noise and a user's voice signal, a touch screen for receiving a user's touch input, a memory for storing a computer program, and a processor, wherein the processor is used to execute the computer program to implement a ticket selling method based on voice recognition as described in any one of claims 1 to 6.

Images