A badcase mining method and electronic device

Abstract

The present application discloses a badcase mining method and electronic device for improving the efficiency of badcase mining. The method comprises: extracting feature information from user log data generated during voice interaction between a smart device and a user; inputting the feature information into the badcase mining model to obtain the category to which the user log data belongs; the category is one of the following categories: a non-badcase data category and at least one badcase data category; the badcase data category is used to indicate that the output of the smart device during voice interaction with the user does not meet the user's expectations; the non-badcase data category is used to indicate that the output of the smart device during voice interaction with the user meets the user's expectations.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a badcase mining method and electronic device. Background Art

[0002] In the process of interaction between smart devices and users, it is usually necessary to perform voice recognition, semantic analysis and other processing on the information input by the user. However, due to the imperfect performance of the semantic engine used for semantic analysis, the output results of the smart device may not meet the user's psychological expectations, that is, a bad case occurs. By analyzing the bad cases that appear, the defects of the semantic engine can be discovered, and then the semantic engine can be improved in a targeted manner to enhance the performance of the semantic engine.

[0003] Currently, bad cases that occur during the interaction between smart devices and users are mainly discovered through manual analysis of user logs. However, manual analysis is time-consuming and labor-intensive, and can often only extract a portion of the massive user logs as samples for analysis. The number of user logs actually analyzed is relatively small, which is not enough to provide assistance for improving the performance of the semantic engine. Summary of the invention

[0004] The embodiments of the present application provide a bad case mining method and an electronic device to improve the efficiency of bad case mining.

[0005] In a first aspect, an electronic device is provided, comprising: a memory and a processor;

[0006] The memory is used to store a pre-trained bad case mining model;

[0007] The processor is configured to:

[0008] Extract feature information from user log data generated during voice interaction between smart devices and users;

[0009] The feature information is input into the badcase mining model to obtain the category to which the user log data belongs; the category is one of the following categories: a non-badcase data category and at least one badcase data category; the badcase data category is used to indicate that the output result of the smart device during the voice interaction process with the user does not meet the user's expectations, and the non-badcase data category is used to indicate that the output result of the smart device during the voice interaction process with the user meets the user's expectations.

[0010] In a second aspect, a badcase mining method is provided, comprising:

[0011] Extract feature information from user log data generated during voice interaction between smart devices and users;

[0012] The feature information is input into a pre-trained badcase mining model to obtain the category to which the user log data belongs; the category is one of the following categories: a non-badcase data category and at least one badcase data category; the badcase data category is used to indicate that the output result of the smart device during the voice interaction process with the user does not meet the user's expectations, and the non-badcase data category is used to indicate that the output result of the smart device during the voice interaction process with the user meets the user's expectations.

[0013] In the above embodiment, the electronic device inputs the user log data into a pre-trained badcase mining model to distinguish the user log data belonging to the badcase data category, thereby realizing the automated mining of badcases and improving the efficiency of badcase mining. The badcase data can be quickly mined from a large amount of user log data, and the obtained mining results can better provide assistance for improving the performance of the semantic engine used to parse semantics. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the implementation methods in the embodiments of the present application or the related technologies, the following is a brief introduction to the drawings required for use in the embodiments or the related technology descriptions. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0015] Figure 1 Schematic diagram of an operation scenario between a display device and a control device according to some embodiments is exemplarily shown in FIG.

[0016] Figure 2 exemplarily shows a hardware configuration block diagram of a display device 200 according to some embodiments;

[0017] Figure 3 exemplarily shows a hardware configuration block diagram of a control device 100 according to some embodiments;

[0018] Figure 4 exemplarily shows a schematic diagram of software configuration in a display device 200 according to some embodiments;

[0019] Figure 5 exemplarily shows a schematic diagram of an icon control interface display of an application in a display device 200 according to some embodiments;

[0020] Figure 6The badcase mining model training flow chart provided in the embodiment of the present application is exemplarily shown in FIG.

[0021] Figure 7 The badcase mining model training diagram provided in the embodiment of the present application is exemplarily shown in FIG.

[0022] Figure 8 The flowchart of the badcase mining method provided in the embodiment of the present application is exemplified in FIG. DETAILED DESCRIPTION

[0023] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0024] Based on the exemplary embodiments described in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the claims attached to this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of the disclosure can also constitute a complete implementation method separately.

[0025] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0026] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise indicated. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented according to orders other than those given in the diagrams or descriptions of the embodiments of this application.

[0027] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0028] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0029] The term "remote control" as used in this application refers to a component of an electronic device (such as the display device disclosed in this application), which can usually wirelessly control the electronic device within a short range. It is generally connected to the electronic device using infrared and / or radio frequency (RF) signals and / or Bluetooth, and may also include functional modules such as WiFi, wireless USB, Bluetooth, motion sensors, etc. For example: a handheld touch remote control replaces most of the physical built-in hard keys in a general remote control device with a user interface in a touch screen.

[0030] The term "gesture" used in this application refers to a user's behavior of expressing an intended idea, action, purpose, or result through a change in hand shape or hand movement.

[0031] In some embodiments below, the smart device that performs voice interaction with the user is described as a display device. For ease of understanding, the structure of the display device, the interaction between the display device and the user, and the interaction between the display device and the server are described below:

[0032] Figure 1 FIG. 2 is a schematic diagram showing an operation scenario between a display device and a control device according to an embodiment. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through the mobile terminal 300 and the control apparatus 100 .

[0033] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, etc., to control the display device 200 wirelessly or in a wired manner. The user may input user commands through buttons on the remote controller, voice input, control panel input, etc. to control the display device 200. For example, the user may input corresponding control commands through the volume plus and minus keys, channel control keys, up / down / left / right movement keys, voice input keys, menu keys, power on / off keys, etc. on the remote controller to realize the function of controlling the display device 200.

[0034] In some embodiments, a mobile terminal, a tablet computer, a computer, a laptop computer, and other smart devices may also be used to control the display device 200. For example, an application running on the smart device is used to control the display device 200. The application may be configured to provide the user with various controls in an intuitive user interface (UI) on a screen associated with the smart device.

[0035] In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and realize connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. For example, it is possible to establish a control instruction protocol with the mobile terminal 300 and synchronize the remote control keyboard to the mobile terminal 300, and realize the function of controlling the display device 200 by controlling the user interface on the mobile terminal 300. It is also possible to transmit the audio and video content displayed on the mobile terminal 300 to the display device 200 to realize the synchronous display function.

[0036] like Figure 1 It is also shown that the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. For example, the display device 200 receives software program updates or accesses a remotely stored digital media library by sending and receiving information and interacting with an electronic program guide (EPG). The server 400 may be a cluster or multiple clusters, and may include one or more types of servers. Other network service content such as video on demand and advertising services is provided through the server 400.

[0037] The display device 200 may be a liquid crystal display, an OLED display, or a projection display device. The specific display device type, size, and resolution are not limited. It is understood by those skilled in the art that the display device 200 may be modified in performance and configuration as required.

[0038] In addition to providing the broadcast receiving television function, the display device 200 may also provide an intelligent network television function with computer support functions, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0039] Figure 2 Schematically shows a hardware configuration block diagram of a display device 200 according to an exemplary embodiment.

[0040] In some embodiments, the display device 200 includes at least one of a controller 250, a tuner and demodulator 210, a communicator 220, a detector 230, an input / output interface 255, a display 275, an audio output interface 285, a memory 260, a power supply 290, a user interface 265, and an external device interface 240.

[0041] In some embodiments, the display 275 is a component for receiving the image signal outputted from the first processor and displaying video content and images as well as a menu control interface.

[0042] In some embodiments, the display 275 includes a display screen component for presenting images and a driving component for driving image display.

[0043] In some embodiments, the displayed video content may be from broadcast television content, or may be various broadcast signals received through wired or wireless communication protocols, or may be from various image content sent from a network server via a network communication protocol.

[0044] In some embodiments, the display 275 is used to present a user manipulation UI interface generated in the display device 200 and used to control the display device 200 .

[0045] In some embodiments, depending on the type of display 275, a driving component for driving the display is also included.

[0046] In some embodiments, the display 275 is a projection display, and may also include a projection device and a projection screen.

[0047] In some embodiments, the communicator 220 is a component for communicating with an external device or an external server according to various communication protocol types. For example, the communicator may include at least one of a Wifi chip, a Bluetooth communication protocol chip, a wired Ethernet communication protocol chip, other network communication protocol chips or a near field communication protocol chip, and an infrared receiver.

[0048] In some embodiments, the display device 200 may establish transmission and reception of control signals and data signals with the external control device 100 or the content providing device through the communicator 220 .

[0049] In some embodiments, the user interface 265 may be used to receive infrared control signals from the control device 100 (eg, an infrared remote controller, etc.).

[0050] In some embodiments, the detector 230 is a signal used by the display device 200 to collect external environment or interact with the outside.

[0051] In some embodiments, the detector 230 includes a light receiver, a sensor for collecting ambient light intensity, and can adaptively display parameter changes by collecting ambient light.

[0052] In some embodiments, the detector 230 may also include an image collector, such as a camera, a webcam, etc., which can be used to collect external environment scenes, as well as to collect user attributes or gestures for interacting with the user. It can adaptively change display parameters and recognize user gestures to achieve the function of interacting with the user.

[0053] In some embodiments, the detector 230 may also include a temperature sensor or the like, such as by sensing the ambient temperature.

[0054] In some embodiments, the display device 200 can adaptively adjust the display color temperature of the image. For example, when the temperature is high, the display device 200 can be adjusted to display a cooler color temperature of the image, or when the temperature is low, the display device 200 can be adjusted to display a warmer color temperature of the image.

[0055] In some embodiments, the detector 230 may also be a sound collector, such as a microphone, which can be used to receive the user's voice, for example, including a voice signal of a control instruction of the user controlling the display device 200, or collect environmental sounds to identify the type of environmental scene, so that the display device 200 can adapt to the environmental noise.

[0056] In some embodiments, Figure 2 As shown, the input / output interface 255 is configured to perform data transmission between the controller 250 and other external devices or other controllers 250, such as receiving video signal data and audio signal data, or command instruction data from external devices.

[0057] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more interfaces such as a high-definition multimedia interface HDMI interface, an analog or digital high-definition component input interface, a composite video input interface, a USB input interface, an RGB port, etc. It is also possible that the above multiple interfaces form a composite input / output interface.

[0058] In some embodiments, Figure 2 As shown, the tuner-demodulator 210 is configured to receive broadcast television signals through wired or wireless reception, and can perform modulation and demodulation processing such as amplification, mixing and resonance, and demodulate audio and video signals from multiple wireless or wired broadcast television signals. The audio and video signals may include television audio and video signals carried in the television channel frequency selected by the user, as well as EPG data signals.

[0059] In some embodiments, the frequency demodulated by the tuner-demodulator 210 is controlled by the controller 250, which can send a control signal based on the user's selection to make the modem respond to the TV signal frequency selected by the user and modulate and demodulate the TV signal carried by the frequency.

[0060] In some embodiments, the broadcast television signal can be classified into terrestrial broadcast signals, cable broadcast signals, satellite broadcast signals, or Internet broadcast signals according to different broadcasting formats of the television signal. Or it can be classified into digital modulation signals, analog modulation signals, etc. according to different modulation types. Or it can be classified into digital signals, analog signals, etc. according to different signal types.

[0061] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box. In this way, the set-top box outputs the TV audio and video signals modulated and demodulated from the received broadcast TV signals to the main device, and the main device receives the audio and video signals through the first input / output interface.

[0062] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 can control the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 275, the controller 250 can perform operations related to the object selected by the user command.

[0063] In some embodiments, the object may be any one of the selectable objects, such as a hyperlink or an icon. Operations related to the selected object may include, for example, displaying a page, document, image, etc. connected to the hyperlink, or executing a program corresponding to the icon. User commands for selecting UI objects may be commands inputted through various input devices (e.g., a mouse, keyboard, touchpad, etc.) connected to the display device 200 or voice commands corresponding to voices spoken by the user.

[0064] like Figure 2 As shown, the controller 250 includes at least one of a random access memory 251 (Random Access Memory, RAM), a read-only memory 252 (Read-Only Memory, ROM), a video processor 270, an audio processor 280, other processors 253 (for example: a graphics processor (Graphics Processing Unit, GPU), a central processing unit 254 (Central Processing Unit, CPU), a communication interface (Communication Interface), and a communication bus 256 (Bus). Among them, the communication bus connects various components.

[0065] In some embodiments, RAM 251 is used to store temporary data for the operating system or other running programs.

[0066] In some embodiments, ROM 252 is used to store various system startup instructions.

[0067] In some embodiments, ROM 252 is used to store a basic input / output system, called Basic Input Output System (BIOS), which is used to complete the power-on self-test of the system, the initialization of each functional module in the system, the basic input / output driver of the system, and the booting operating system.

[0068] In some embodiments, when a power-on signal is received, the display device 200 starts to power on, and the CPU runs the system startup instruction in the ROM 252, and copies the temporary data of the operating system stored in the memory to the RAM 251, so as to start or run the operating system. After the operating system is started, the CPU copies the temporary data of various application programs in the memory to the RAM 251, so as to start or run various application programs.

[0069] In some embodiments, the CPU processor 254 is used to execute operating system and application instructions stored in the memory, and to execute various applications, data and content according to various interactive instructions received from external input, so as to finally display and play various audio and video contents.

[0070] In some exemplary embodiments, the CPU processor 254 may include multiple processors. The multiple processors may include a main processor and one or more sub-processors. The main processor is used to perform some operations of the display device 200 in the pre-power-on mode and / or the operation of displaying the picture in the normal mode. The one or more sub-processors are used for an operation in the standby mode and the like.

[0071] In some embodiments, the graphics processor 253 is used to generate various graphic objects, such as icons, operation menus, and user input command display graphics, etc. It includes an operator that performs operations by receiving various interactive instructions input by the user and displays various objects according to display attributes. It also includes a renderer that renders various objects obtained based on the operator, and the rendered objects are used to be displayed on the display.

[0072] In some embodiments, the video processor 270 is configured to receive an external video signal and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image synthesis, etc. according to the standard codec protocol of the input signal, and obtain a signal that can be directly displayed or played on the display device 200.

[0073] In some embodiments, the video processor 270 includes a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, etc.

[0074] Among them, the demultiplexing module is used to demultiplex the input audio and video data stream. For example, if MPEG-2 is input, the demultiplexing module demultiplexes it into video signals and audio signals.

[0075] The video decoding module is used to process the demultiplexed video signal, including decoding and scaling.

[0076] The image synthesis module, such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator according to the user input or itself with the scaled video image to generate an image signal for display.

[0077] The frame rate conversion module is used to convert the input video frame rate, such as converting the 60Hz frame rate to 120Hz frame rate or 240Hz frame rate. The usual format is implemented by interpolation.

[0078] The display formatting module is used to receive the frame rate converted video output signal and change the signal to conform to the display format signal, such as outputting RGB data signal.

[0079] In some embodiments, the graphics processor 253 can be integrated with the video processor or can be separately configured. When integrated, it can perform processing of graphic signals output to the display. When separately configured, it can perform different functions respectively, such as a GPU+FRC (Frame Rate Conversion) architecture.

[0080] In some embodiments, the audio processor 280 is used to receive an external audio signal, and perform decompression and decoding, as well as noise reduction, digital-to-analog conversion, and amplification processing according to the standard codec protocol of the input signal to obtain a sound signal that can be played in a speaker.

[0081] In some embodiments, the video processor 270 may include one or more chips. The audio processor may also include one or more chips.

[0082] In some embodiments, the video processor 270 and the audio processor 280 may be integrated into separate chips, or may be integrated into one or more chips together with the controller.

[0083] In some embodiments, the audio output receives the sound signal output by the audio processor 280 under the control of the controller 250, such as the speaker 286, and in addition to the speaker carried by the display device 200 itself, can be output to the external audio output terminal of the generating device of the external device, such as an external audio interface or a headphone interface, etc., and can also include a short-range communication module in the communication interface, for example, a Bluetooth module for Bluetooth speaker sound output.

[0084] The power supply 290, under the control of the controller 250, uses the power input from the external power supply to provide power supply support for the display device 200. The power supply 290 may include a built-in power supply circuit installed inside the display device 200, or may be an external power supply installed in the display device 200, providing a power supply interface for an external power supply in the display device 200.

[0085] The user interface 265 is used to receive a user input signal, and then send the received user input signal to the controller 250. The user input signal may be a remote control signal received by an infrared receiver, and may receive various user control signals through a network communication module.

[0086] In some embodiments, the user inputs a user command through the control device 100 or the mobile terminal 300 , the user input interface is based on the user input, and the display device 200 responds to the user input through the controller 250 .

[0087] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 275, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.

[0088] In some embodiments, the "user interface" is a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, and other visual interface elements.

[0089] The memory 260 includes storing various software modules for driving the display device 200. For example, the various software modules stored in the first memory include at least one of a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.

[0090] The basic module is a bottom-level software module used for signal communication between various hardware in the display device 200 and sending processing and control signals to the upper-level modules. The detection module is a management module used to collect various information from various sensors or user input interfaces, and perform digital-to-analog conversion and analysis management.

[0091] For example, the speech recognition module includes a speech analysis module and a speech instruction database module. The display control module is a module for controlling the display to display image content, and can be used to play multimedia image content and UI interface information. The communication module is a module for controlling and communicating data with external devices. The browser module is a module for executing data communication between browsing servers. The service module is a module for providing various services and various applications. At the same time, the memory 260 is also used to store and receive external data and user data, images of various items in various user interfaces, and visual effects of focus objects.

[0092] Figure 3 Schematically shows a block diagram of a configuration of a control device 100 according to an exemplary embodiment. Figure 3 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0093] The control device 100 is configured to control the display device 200, and can receive the user's input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200. For example, the user operates the channel increase and decrease keys on the control device 100, and the display device 200 responds to the channel increase and decrease operations.

[0094] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0095] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other smart electronic device can play a similar function to the control device 100 after installing the application for controlling the display device 200. For example, the user can install the application to realize the function of the physical button of the control device 100 through various function keys or virtual buttons of the graphical user interface provided on the mobile terminal 300 or other smart electronic device.

[0096] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0097] The communication interface 130, under the control of the controller 110, realizes the communication of control signals and data signals with the display device 200. For example, the received user input signal is sent to the display device 200. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0098] The user input / output interface 140 includes at least one of other input interfaces such as a microphone 141, a touch panel 142, a sensor 143, and a button 144. For example, the user can input user commands through voice, touch, gesture, pressing, and other actions. The input interface converts the received analog signal into a digital signal, and the digital signal into a corresponding command signal, and sends it to the display device 200.

[0099] The output interface includes an interface for sending received user commands to the display device 200. In some embodiments, it can be an infrared interface or a radio frequency interface. For example, in the case of an infrared signal interface, the user input command needs to be converted into an infrared control signal according to an infrared control protocol and sent to the display device 200 via an infrared sending module. Another example is in the case of a radio frequency signal interface, the user input command needs to be converted into a digital signal, which is then modulated according to a radio frequency control signal modulation protocol and sent to the display device 200 via a radio frequency sending terminal.

[0100] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, which can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0101] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 200 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0102] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller, and can be a battery and related control circuits.

[0103] In some embodiments, the system may include a kernel, a command parser (shell), a file system, and an application. The kernel, shell, and file system together form the basic operating system structure, which allows users to manage files, run programs, and use the system. After power-on, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, schedulers, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. After startup, the application is compiled into machine code to form a process.

[0104] See also Figure 4 In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0105] In some embodiments, at least one application is running in the application layer, and these applications may be window programs, system settings programs, clock programs, camera applications, etc. provided by the operating system; or they may be applications developed by third-party developers, such as hi-view programs, karaoke programs, magic mirror programs, etc. In specific implementation, the application packages in the application layer are not limited to the above examples, and may actually include other application packages, which are not limited in the embodiments of the present application.

[0106] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Applications can access system resources and obtain system services during execution through the API interface.

[0107] like Figure 4As shown, in the embodiment of the present application, the application framework layer includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager (ActivityManager) is used to interact with all activities running in the system; a location manager (Location Manager) is used to provide system services or applications with access to system location services; a package manager (Package Manager) is used to retrieve various information related to the application package currently installed on the device; a notification manager (NotificationManager) is used to control the display and clearing of notification messages; a window manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0108] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0109] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, touch sensor, pressure sensor, etc.), etc.

[0110] In some embodiments, the core layer further includes a power driver module for performing power management.

[0111] In some embodiments, Figure 4 The software programs and / or modules corresponding to the software architecture in Figure 2 or Figure 3 in the first memory or the second memory shown.

[0112] In some embodiments, Figure 5 As shown in , the application layer includes at least one application that can display a corresponding icon control in the display, such as: a live TV application icon control, a video on demand application icon control, a media center application icon control, an application center icon control, a game application icon control, etc.

[0113] In some embodiments, a bad case refers to a problem in which the output of a smart device such as a smart TV, smart speaker, or smart phone does not meet the user's expectations during the interaction between the smart device and the user. For example, if a user inputs "I want to listen to Sweet Honey" into a smart device and expects the smart device to play the song Sweet Honey, but the smart device outputs the TV series Sweet Honey, this output of the smart device obviously does not meet the user's expectations, and a bad case can be considered to have occurred.

[0114] By analyzing the bad cases that occur, we can find out the defects of the semantic engine used to parse semantics during the interaction between smart devices and users, so that we can improve the semantic engine in a targeted manner and enhance its performance. However, currently, the bad cases that occur during the interaction between smart devices and users are mainly discovered by manually analyzing user log data. This method is time-consuming and labor-intensive, and often only a portion of the massive user logs can be extracted as samples for analysis. The number of user logs actually analyzed is relatively small, which is not enough to provide assistance for improving the performance of the semantic engine.

[0115] To this end, the embodiment of the present application provides a badcase mining method, which can efficiently mine badcases from a large amount of user log data by using a badcase mining model, thereby providing assistance for improving the performance of the semantic engine. The badcase data here refers to data whose output results of the corresponding smart device do not meet the user's expectations, that is, data corresponding to badcases.

[0116] For ease of description, the embodiments of the present application are mainly described based on the voice interaction between the smart device and the user. In actual application, if the user adopts other methods to interact with the smart device, such as the user inputs text sentences to the smart device through a keyboard or control device, the relevant processes can be processed differently.

[0117] To facilitate understanding of the implementation of the badcase mining method provided in the embodiment of the present application, the following first combines Figure 6 Describe how to train the badcase mining model:

[0118] See also Figure 6 , Figure 6 The badcase mining model training flow chart provided by the embodiment of the present application is exemplarily shown in FIG. Figure 6 As shown, the process may include the following steps:

[0119] Step S51: obtaining a training sample, where the training sample includes user log data of labeled categories.

[0120] In one example, user log data is generated based on the voice interaction process between the smart device and the user. In one embodiment, during the voice interaction process between the smart device and the user, the user's voice input sentence (also referred to as the user's input voice) needs to be processed by voice recognition, semantic parsing, etc. The voice interaction process between the smart device and the user will be described by example later, so it will not be repeated here.

[0121] As an embodiment, for any sentence input by user voice into the smart device, the content recorded in the user log data may include: the speech recognition result corresponding to the sentence, the semantic analysis result corresponding to the sentence, the output result of the smart device in response to the sentence and / or the user stay time corresponding to the output result, etc. For the convenience of description, in the following embodiments, unless otherwise specified, the sentence input by user voice into the smart device is described as a query sentence (query).

[0122] In one example, the categories annotated for the user log data in step S51 may include: badcase data categories and non-badcase data categories. The badcase data category is used to indicate that the output result of the smart device during the voice interaction with the user does not meet the user's expectations, and the non-badcase data category is used to indicate that the output result of the smart device during the voice interaction with the user meets the user's expectations. As an embodiment, the category to which the user log data in the training sample belongs can be manually annotated, and different numerical values ​​can be used to represent different categories during specific annotation.

[0123] Step S52: Use the above training samples to train the badcase mining model.

[0124] As an embodiment, the badcase mining model can be a machine learning model, such as a support vector machine (SVM) model, a tree-based machine learning model such as a random forest model, etc., or a neural network model, a deep neural network model (Deep Neural Networks, DNN) such as a convolutional neural network (CNN), etc.

[0125] In one example, the following steps a1-a2 may be used to iteratively train the badcase mining model until the training completion condition is met:

[0126] Step a1, extracting feature information from the training sample and inputting it into the badcase mining model to obtain the prediction result output by the badcase mining model;

[0127] Step a2: update the parameters of the badcase mining model according to the prediction results output by the badcase mining model and the categories marked in the training samples.

[0128] In one example, the feature information in step a1 is a feature that can distinguish badcase data categories from non-badcase data categories. As an embodiment, which features of the user log data are used as feature information can be determined by feature engineering, which can be accomplished by manually analyzing the user log data.

[0129] As an embodiment, for the user log data recorded corresponding to any query statement input by the user (hereinafter referred to as the first statement for ease of description), the above-mentioned feature information may include: the semantic analysis result corresponding to the first statement, the semantic analysis result corresponding to the next query statement of the first statement, the semantic similarity between the first statement and the adjacent query statement (such as the similarity of the phonetic code), the output result of the smart device in response to the first statement and / or the user's stay time corresponding to the output result, etc. As an embodiment, the above-mentioned next query statement is the query statement input by the user into the smart device after the first statement, and the above-mentioned adjacent query statement is the query statement input by the user into the smart device before and / or after the first statement. As an embodiment, the semantic analysis result corresponding to the first statement in the above-mentioned feature information may include the business field, intention, etc. of the first statement obtained by semantically analyzing the first statement.

[0130] The following is a brief description of how some of the above features are used to distinguish badcase data categories from non-badcase data categories from user log data, that is, how some of the above features reflect whether the output results of the smart device meet the user's expectations:

[0131] 1) User stay time:

[0132] As an example, when the output result of the smart device in response to the first sentence meets the user's expectations, the user will stay under the output result for a certain period of time to view the output result, and when the output result does not meet the user's expectations, the user may quickly leave the output result. Therefore, the length of the user's stay time can reflect to a certain extent whether the output result of the smart device meets the user's expectations.

[0133] 2) Similarity between the first sentence and the adjacent query sentence:

[0134] As an embodiment, if the similarity between the first sentence and the adjacent query sentence is high, it means that the user may have inputted semantically similar query sentences for many times, which further means that the user may not get the expected output result. Therefore, the similarity between the first sentence and the adjacent query sentence can reflect to some extent whether the output result of the smart device meets the user's expectations.

[0135] After extracting feature information from the training samples, as an embodiment, the text data in the feature information may be converted into numerical data, a feature vector may be generated based on the feature information consisting entirely of numerical data, and the feature vector may be input into the badcase mining model.

[0136] In one example, when the badcase mining model meets the training completion condition, the training is stopped.

[0137] As an embodiment, the above training completion condition may include: the prediction accuracy of the badcase mining model is greater than the first threshold. As an embodiment, when the loss function of the badcase mining model is less than the second threshold, it can be determined that the prediction accuracy of the badcase mining model is greater than the first threshold. When the training completion condition is met, the iterative training of the badcase mining model can be stopped to obtain a trained badcase mining model.

[0138] So far, the Figure 6 Description of the process shown.

[0139] pass Figure 6 The process shown is used to train a badcase mining model, which can more accurately predict the category to which user log data belongs, thereby efficiently determining the data belonging to the badcase data category in a large amount of user log data.

[0140] In one example, the badcase data category can be further divided into multiple categories. As an embodiment, the badcase data category can be divided into: new demand problem data, identification problem data, error correction problem data and positioning problem data according to the cause of the badcase, and these four types of data correspond to badcases caused by different reasons.

[0141] As an embodiment, the badcase corresponding to the above new demand problem data is a badcase caused by the intention of the query statement entered by the user that is not currently supported by the semantic engine. For example, if the intention of the query statement entered by the user is "delete the play record", if the semantic engine currently does not support the parsing of this intention, the intention of the query statement cannot be correctly parsed, resulting in the smart device not executing the operation of deleting the play record, resulting in a badcase.

[0142] As an embodiment, the badcase corresponding to the above-mentioned recognition problem data is a badcase caused by an error in the process of recognizing the query sentence input by the user's voice as text. For example, the query sentence actually input by the user's voice is "epidemic prevention and control ideological and political course", and when the query sentence is recognized as text, if it is mistakenly recognized as "epidemic prevention and control four certificates big customer", it will cause a badcase.

[0143] As an embodiment, the badcase corresponding to the above error correction problem data is a badcase caused by failure to correct the erroneous information in the query statement input by the user. When the user enters the query statement, due to incorrect memory of the resource name, inability to express their own needs smoothly, use of dialects, etc., the query statement actually entered may contain erroneous information that does not meet the user's actual needs. If the semantic engine fails to correct the erroneous information, it may cause a badcase. For example, the user wants to watch the film "Peach Blossoms in the Enclosed House", but mistakenly enters "Peach Blossoms in the Enclosed Wall". If the semantic engine fails to correct the error, it may cause a badcase.

[0144] As an embodiment, the bad case corresponding to the above positioning problem data is a bad case caused by incorrect positioning of the business field of the query statement input by the user. For example, the query statement input by the user is "someone's resume". For this query statement, the user actually expects to search in the encyclopedia field. If the semantic parsing engine fails to locate the statement in the encyclopedia field, but locates it in other business fields such as the music field, a bad case may be caused.

[0145] According to the above classification of badcase data categories, as an embodiment, the badcase mining model can be enabled to distinguish user log data belonging to the new demand problem data category, the identification problem data category, the error correction problem data category, the positioning problem data category and the non-badcase data category. The training process of the badcase mining model can be found in Figure 7 shown.

[0146] Figure 7 The badcase mining model training diagram provided in the embodiment of the present application is exemplarily shown in FIG.

[0147] like Figure 7As shown, the categories annotated by the user log data in the training sample include five categories: new demand problem data category, identification problem data category, error correction problem data category, location problem data category and non-badcase data category. The badcase mining model is trained using the training sample until the badcase mining model meets the training completion conditions. During the training process, the feature information extracted from the training sample is feature information that can distinguish the above five categories. For details, please refer to the feature information exemplified above, and no further examples will be given here. Figure 7 In the training process shown in FIG. 1 , the badcase mining model can more accurately distinguish the user log data belonging to the above five categories.

[0148] It should be noted that the above classification of badcase data categories and the badcase mining model trained according to multiple badcase data categories are only examples and are not used to limit the present application. When applied, the badcase data categories can be classified differently from the above embodiment according to actual needs, and the badcase mining model can be trained according to the classified multiple badcase data categories, which will not be described in detail here.

[0149] So far, the description of the badcase mining model training process provided by the embodiment of the present application is completed. Based on the trained badcase mining model, the embodiment of the present application provides a badcase mining method. Figure 8 Describe:

[0150] See also Figure 8 , Figure 8 The flowchart of the badcase mining method provided by the embodiment of the present application is exemplarily shown in FIG. In one example, the method can be applied to electronic devices such as servers, computers, etc. As an embodiment, the electronic device includes at least a memory and a processor. When the badcase mining method provided by the embodiment of the present application is applied, the memory can be used to store a pre-trained badcase mining model, and the processor can be used to execute related programs to implement the badcase mining method.

[0151] like Figure 8 As shown, the badcase mining method provided in the embodiment of the present application may include the following steps:

[0152] Step S61, extracting feature information from user log data.

[0153] Different from the user log data used as training samples in the above step S51, the user log data in this step S61 is user log data without a labeled category.

[0154] For ease of description, the following describes Figure 8 In the process shown, the device applying the badcase mining method provided by the embodiment of the present application is recorded as this device. In one example, this device can be a device that generates user log data based on the voice interaction process between the smart device and the user (hereinafter referred to as the log device), or it can be not a log device.

[0155] If the device is a log device, as an embodiment, the newly generated user log data of the device can be obtained in real time, or the user log data generated by the device within a certain period of time (such as one day or one week) can be obtained.

[0156] If the device is not a log device, the user log data can be obtained from the log device. In one embodiment, the user log data generated by the log device is recorded in a log file, and the device can monitor the log file of the log device, and when the log device records new user log data in the log file, the user log data is obtained. In another embodiment, the device can obtain the user log data generated within a certain period of time from the log device at regular intervals.

[0157] The feature information extracted from the user log data in step S61 can refer to the feature information extracted from the training samples when training the badcase mining model above, and will not be described here with examples.

[0158] Step S62: input the above feature information into a pre-trained badcase mining model to obtain the category to which the above user log data belongs.

[0159] The category to which the user log data obtained in this step S62 belongs may be one of the following categories: a non-badcase data category and at least one badcase data category, wherein at least one badcase data category may include at least one of new demand problem data, identification problem data, error correction problem data, and positioning problem data, and may also include other badcase data categories.

[0160] As an embodiment, if the prediction result output by the badcase mining model is the probability that the user log data belongs to each category, the category to which the user log data belongs can be further determined based on the probability. For example, the category corresponding to the highest probability is determined as the category to which the user log data belongs, and the category corresponding to the probability exceeding the threshold is determined as the category to which the user log data belongs, and so on.

[0161] So far, the Figure 8 Description of the process shown.

[0162] pass Figure 8As shown in the process, the embodiment of the present application can use the pre-trained badcase mining model to mine the badcase data in the user log data, and then determine the badcase in the voice interaction process between the user and the smart device. By analyzing the badcase, the defects of the semantic engine can be found, so as to improve the semantic engine in a targeted manner and enhance the performance of the semantic engine.

[0163] As an example, Figure 8 In the process shown, the prediction results output by the badcase mining model are manually verified, and the output results verified to be correct are used as training data for the badcase mining model, which are used to train the badcase mining model to update the badcase mining model, so that the prediction results output by the badcase mining model are more accurate. As an embodiment, the above process of updating the badcase mining model can be performed periodically.

[0164] To better understand this application, the following takes a smart device as a display device as an example. Figure 1 The following operation scenarios provide examples of the voice interaction process between smart devices and users:

[0165] like Figure 1 As shown, the operation scenario mainly involves a user, a control device, a display device 200 and a server 400. As described above, the display device 200 can communicate data with the server 400 through a variety of communication methods, wherein the server 400 can be a cluster or multiple clusters, and can include one or more types of servers.

[0166] In one embodiment, a speech recognition service and a semantic engine may be deployed on the server or cluster included in the server 400 , wherein the speech recognition service is used to recognize audio as text, and the semantic engine is used to parse the semantics of the text.

[0167] Taking the example of a first server included in server 400 on which a speech recognition service and a semantic engine are deployed simultaneously, after receiving a query statement input by a user's voice, the display device 200 can denoise the audio of the query statement and upload the processed audio to the above-mentioned first server so that the speech recognition service deployed by the first server can be used to recognize the audio as text.

[0168] For the recognized text, the first server can use the deployed semantic engine to perform semantic analysis on the text to obtain a semantic analysis result, which may include the business field, intention, etc. of the text. Based on the obtained semantic analysis result, the first server can obtain the query result of the query statement from the corresponding business server. For example, if the query statement wants to query today's weather, the first server can obtain today's weather information from the server providing weather services as the query result of the query statement.

[0169] After obtaining the query result, the first server may send the query result obtained and information such as feedback statements in the semantic analysis result to the display device 200, so that the display device 200 outputs the information through voice broadcast or other means.

[0170] At this point, the description of the voice interaction process between the user and the display device is completed.

[0171] It should be noted that the above-mentioned voice interaction process and operation scenarios are only examples and are not intended to limit the scope of protection of this application.

[0172] In one example, the first server may generate user log data based on the voice interaction process between the user and the smart device. As an embodiment, for a query statement input by the user, the first server may record the voice recognition result, semantic parsing result, query result returned to the smart device, etc. corresponding to the statement. As an embodiment, the first server may also generate user log data based on information such as the time the user stays under the output result reported by the smart device.

[0173] In the above embodiments, the electronic device can mine badcase data from user log data through a pre-trained badcase mining model, thereby efficiently mining badcases from a large number of user logs, effectively saving manpower and time, and the obtained analysis results can better provide assistance for improving the performance of the semantic engine.

[0174] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0175] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. An electronic device, characterized in that: include: Memory and processor; The memory is used to store a pre-trained bad case mining model; wherein the bad case mining model is trained by the following steps: Obtain training samples, wherein the training samples include user log data with labeled categories; the labeled categories include five categories: new demand problem data category, identification problem data category, error correction problem data category, positioning problem data category, and non-badcase data category; Using the training samples to train the badcase mining model; The processor is configured to: Extracting feature information from user log data generated during voice interaction between a smart device and a user; wherein the user log includes at least one of the following information: The speech recognition result corresponding to each speech input by the user during the speech interaction between the smart device and the user, the semantic analysis result corresponding to the speech, the output result of the smart device in response to the speech, and the user stay time corresponding to the output result; The characteristic information includes at least one of the following information: The semantic analysis result corresponding to the speech, the output result of the smart device in response to the speech, the user's stay time corresponding to the output result, the semantic analysis result corresponding to the next speech of the speech, and the semantic similarity between the speech and the next speech; Input the feature information into the badcase mining model to obtain the category to which the user log data belongs; the category is one of the following categories: a non-badcase data category and at least one badcase data category; the badcase data category is used to indicate that the output result of the smart device during the voice interaction with the user does not meet the user's expectations, and the non-badcase data category is used to indicate that the output result of the smart device during the voice interaction with the user meets the user's expectations; Wherein, the at least one badcase data category includes at least one of the following: new demand problem data, identification problem data, error correction problem data, and location problem data; The badcase corresponding to the new demand question data is a badcase caused by the intention of the query statement input by the user being an intention that the semantic engine currently does not support parsing; The badcase corresponding to the recognition problem data is a badcase caused by an error in the process of recognizing the query sentence input by the user's voice as text; The badcase corresponding to the error correction problem data is a badcase caused by failure to correct the error information in the query statement input by the user; The badcase corresponding to the positioning problem data is a badcase caused by incorrect positioning of the business domain of the query statement input by the user.

2. The electronic device according to claim 1, characterized in that: The badcase mining model is one of a support vector machine model, a tree-structured machine learning model, a neural network model and a deep neural network model.

3. The electronic device according to claim 1, characterized in that: The semantic analysis result corresponding to the speech is obtained in the following way: Perform speech recognition on the speech to obtain text, and perform semantic analysis on the text to obtain corresponding semantic analysis results.

4. A badcase mining method, characterized in that: include: Extracting feature information from user log data generated during voice interaction between a smart device and a user; wherein the user log includes at least one of the following information: The speech recognition result corresponding to each speech input by the user during the speech interaction between the smart device and the user, the semantic analysis result corresponding to the speech, the output result of the smart device in response to the speech, and the user stay time corresponding to the output result; The characteristic information includes at least one of the following information: The semantic analysis result corresponding to the speech, the output result of the smart device in response to the speech, the user's stay time corresponding to the output result, the semantic analysis result corresponding to the next speech of the speech, and the semantic similarity between the speech and the next speech; The feature information is input into a pre-trained badcase mining model to obtain a category to which the user log data belongs; the category is one of the following categories: a non-badcase data category and at least one badcase data category; the badcase data category is used to indicate that the output result of the smart device during the voice interaction with the user does not meet the user's expectations, and the non-badcase data category is used to indicate that the output result of the smart device during the voice interaction with the user meets the user's expectations; Wherein, the at least one badcase data category includes at least one of the following: new demand problem data, identification problem data, error correction problem data, and location problem data; The badcase corresponding to the new demand question data is a badcase caused by the intention of the query statement input by the user being an intention that the semantic engine currently does not support parsing; The badcase corresponding to the recognition problem data is a badcase caused by an error in the process of recognizing the query sentence input by the user's voice as text; The badcase corresponding to the error correction problem data is a badcase caused by failure to correct the error information in the query statement input by the user; The badcase corresponding to the positioning problem data is a badcase caused by an error in positioning the business domain of the query statement input by the user; The badcase mining model is trained by the following steps: Acquire training samples, wherein the training samples include user log data with labeled categories; the labeled categories include five categories: new demand problem data category, identification problem data category, error correction problem data category, positioning problem data category and non-badcase data category; use the training samples to train the badcase mining model.

Images