Problem feedback method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application provide a problem feedback method and device, electronic equipment and storage medium, belonging to the technical field of computer. The method comprises: in response to a first interaction event of a terminal, displaying a problem feedback interface corresponding to the first interaction event; in response to a second interaction event triggered on the problem feedback interface, obtaining feedback voice information, performing voice recognition on the feedback voice information to obtain first text information; performing semantic analysis on the first text information to obtain a semantic analysis result, calling back a corresponding application interface according to the semantic analysis result, and performing image capture on the application interface to obtain first image information; generating a problem feedback file according to the first text information and the first image information, and uploading the problem feedback file to a server. The embodiments of the present application can improve the efficiency of problem feedback.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a problem feedback method and device, electronic equipment and storage medium. BACKGROUND

[0002] The feedback of users is very important for the development of products, which can help the product developers to better understand the problems encountered by users, the functions expected to be added and a series of other suggestions, so that the product developers have a more comprehensive understanding of their products and have important guiding significance for future product planning.

[0003] In the current application products, users often need to spend a lot of time to find the problem feedback entrance, and then input the feedback problems through screenshot, input a large amount of text and other ways, and submit the feedback content according to the guided step-by-step operation. The whole process is complex and cumbersome, and the efficiency is low, which affects the user experience of product use, so that a large part of users, especially the elderly, are unwilling to feedback problems or give up feedback in the operation. Therefore, how to improve the efficiency of problem feedback of users when using application products has become a technical problem to be solved. SUMMARY

[0004] The main purpose of the embodiments of the present application is to provide a problem feedback method and device, electronic equipment and storage medium, which aims to improve the efficiency of problem feedback.

[0005] To achieve the above purpose, the first aspect of the embodiments of the present application provides a problem feedback method, which comprises:

[0006] In response to a first interaction event of a terminal, a problem feedback interface corresponding to the first interaction event is displayed;

[0007] In response to a second interaction event triggered on the problem feedback interface, feedback voice information is obtained, and first text information is obtained by performing voice recognition on the feedback voice information;

[0008] The first text information is subjected to semantic analysis to obtain a semantic analysis result, a corresponding application interface is recalled according to the semantic analysis result, and first image information is obtained by performing image interception on the application interface;

[0009] A problem feedback file is generated according to the first text information and the first image information, and the problem feedback file is uploaded to a server.

[0010] In some embodiments, in response to the first interaction event of the terminal, the problem feedback interface corresponding to the first interaction event is displayed, which comprises:

[0011] The interaction identifier of the first interaction event is determined;

[0012] Match the corresponding problem feedback interface in the preset problem feedback interface library according to the interaction identifier;

[0013] The problem feedback interface is displayed on the terminal.

[0014] In some embodiments, the step of responding to a second interactive event triggered on the problem feedback interface, obtaining feedback voice information, and performing speech recognition on the feedback voice information to obtain first text information includes:

[0015] When a first touch operation triggered by the first area is detected, the terminal records voice of the current environment.

[0016] When a second touch operation triggered by the first area is detected, the terminal stops voice recording and obtains the feedback voice information;

[0017] The feedback voice information is input into a pre-trained speech recognition model to obtain candidate text information, which is then displayed through the question feedback interface.

[0018] When a third touch operation triggered by the second area is detected, return to the step of recording voice of the current environment through the terminal;

[0019] When a fourth touch operation triggered by the second area is detected, the candidate text information is determined to be the first text information;

[0020] Both the first area and the second area are located on the problem feedback interface.

[0021] In some embodiments, the step of performing semantic analysis on the first text information to obtain a semantic analysis result, calling back the corresponding application interface based on the semantic analysis result, and capturing an image of the application interface to obtain the first image information includes:

[0022] The first text information is input into a pre-trained semantic analysis model to obtain the semantic analysis results;

[0023] Obtain the current process list of the terminal, and traverse the current process list according to the semantic analysis results to determine the corresponding first application process;

[0024] The terminal displays the application interface of the first application process, and the application interface is captured according to the preset image capture template to obtain the first image information.

[0025] The image cropping template includes a pre-set image cropping border, occlusion area size, and occlusion area position.

[0026] In some embodiments, generating a problem feedback file based on the first text information and the first image information includes:

[0027] Based on the semantic analysis results, information is extracted from the first text information to obtain several keywords;

[0028] The first image information is subjected to OCR text recognition, and the first image information is labeled according to the text recognition result and the keywords to obtain the second image information;

[0029] The first text information and the second image information are packaged and processed to generate a problem feedback file.

[0030] In some embodiments, generating a problem feedback file based on the first text information and the first image information includes:

[0031] Based on the semantic analysis results, information is extracted from the first text information to obtain several keywords;

[0032] The first image information is subjected to OCR text recognition, and the first image information is labeled according to the text recognition result and the keywords to obtain the second image information;

[0033] Obtain the system log information of the terminal;

[0034] The first text information, the second image information, and the system log information are packaged and processed to generate a problem feedback file;

[0035] The system log information includes at least one of the following: user operation path, CPU utilization, memory utilization, disk space utilization, and network status.

[0036] In some embodiments, the first interaction event includes at least one of the following: a shaking event, a rotation event, a pressing event, a wrist raising event, a wrist flipping event, a head lowering event, a head raising event, a head shaking event, a voice input event, and a motion sensing event;

[0037] The second interaction event includes at least one of the following: click event, long press event, zoom event, swipe event, and drag event.

[0038] To achieve the above objectives, a second aspect of this application provides a problem feedback device, the device comprising:

[0039] The interface display module is configured to respond to the first interactive event of the terminal and display the problem feedback interface corresponding to the first interactive event.

[0040] The speech recognition module is configured to respond to a second interactive event triggered on the problem feedback interface, acquire feedback speech information, and perform speech recognition on the feedback speech information to obtain first text information.

[0041] The image capture module is configured to perform semantic analysis on the first text information to obtain the semantic analysis result, call back the corresponding application interface based on the semantic analysis result, and capture the first image information by performing image capture on the application interface.

[0042] The file upload module is configured to generate a problem feedback file based on the first text information and the first image information, and upload the problem feedback file to the server.

[0043] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, it implements the problem feedback method described in the first aspect above.

[0044] To achieve the above objectives, a fourth aspect of the present application provides a storage medium, which is a computer-readable storage medium for computer-readable storage, wherein the storage medium stores one or more programs that can be executed by one or more processors to implement the problem feedback method as described in the first aspect above.

[0045] The problem feedback method, apparatus, electronic device, and storage medium proposed in this application display a corresponding problem feedback interface in response to a first interactive event on a terminal, acquire feedback voice information in response to a second interactive event triggered on the problem feedback interface, perform speech recognition on the feedback voice information to obtain first text information, perform semantic analysis on the first text information to obtain semantic analysis results, call back the corresponding application interface based on the semantic analysis results and perform image capture to obtain first image information, and finally generate a problem feedback file based on the first text information and the first image information and upload it to a server. This application embodiment directly displays the corresponding problem feedback interface through the terminal's interactive events, eliminating the need for users to search for feedback entry points in the application interface, thus improving the efficiency of problem feedback; acquiring feedback voice information and performing speech recognition through interactive events triggered on the problem feedback interface eliminates the need for user input, saving user operation time and further improving the efficiency of problem feedback; by performing semantic analysis on the recognized text information and calling back the corresponding application interface based on the semantic analysis results to perform image capture, it can automatically acquire image information of the application interface where the problem occurs, improving the comprehensiveness and real-time nature of problem feedback, further enhancing the efficiency of problem feedback. Attached Figure Description

[0046] Figure 1 This is a flowchart of the problem feedback method provided in the embodiments of this application;

[0047] Figure 2 yes Figure 1 The flowchart of step S101 in the text;

[0048] Figure 3 yes Figure 1 The flowchart of step S102 in the document;

[0049] Figure 4 yes Figure 1 The flowchart of step S103 in the process;

[0050] Figure 5 yes Figure 1 A flowchart of one embodiment of step S104 in the process;

[0051] Figure 6 yes Figure 1 A flowchart of another implementation of step S104 in the process;

[0052] Figure 7 This is a schematic diagram of the problem feedback device provided in the embodiments of this application;

[0053] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0057] First, let's analyze some of the terms used in this application:

[0058] Human-Computer Interaction (HCI) is a discipline that studies the interaction between a system and a user. The system can be various types of machines, as well as computerized systems and software. The HCI interface typically refers to the user-visible part, through which the user interacts with and operates the system. Examples range from small things like a radio's play button to large things like an airplane's instrument panel or a power plant's control room. A HCI event refers to the information exchange process between a human and a computer using a specific dialogue language and interactive method to complete a defined task.

[0059] Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0060] Natural Language Processing (NLP): NLP uses computers to process, understand, and utilize human language (such as Chinese and English). NLP is a branch of artificial intelligence and an interdisciplinary field of computer science and linguistics, often referred to as computational linguistics. NLP includes syntactic analysis, semantic analysis, and discourse understanding. It is commonly used in machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, intent recognition, information extraction and filtering, text classification and clustering, sentiment analysis, and opinion mining. It involves data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computation.

[0061] Information extraction is a text processing technique that extracts factual information such as entities, relationships, and events from natural language text and outputs it as structured data. Information extraction is a technique for extracting specific information from text data. Text data is composed of specific units, such as sentences, paragraphs, and chapters. Text information is composed of smaller, specific units, such as characters, words, phrases, sentences, paragraphs, or combinations of these units. Extracting noun phrases, names of people, and place names from text data is an example of text information extraction. Of course, text information extraction techniques can extract information of various types.

[0062] OCR (Optical Character Recognition) refers to the process by which electronic devices (such as scanners or digital cameras) examine printed characters on paper, determine their shapes by detecting dark and light patterns, and then translate these shapes into computer text using character recognition methods. In other words, for printed characters, it uses optical methods to convert the text in a paper document into a black-and-white dot matrix image file, and then uses recognition software to convert the text in the image into text format for further editing by word processing software. How to correct errors or utilize auxiliary information to improve recognition accuracy is the most important issue in OCR. The main indicators for evaluating the performance of an OCR system include: rejection rate, false recognition rate, recognition speed, user interface friendliness, product stability, ease of use, and feasibility.

[0063] Based on this, embodiments of this application provide a problem feedback method and apparatus, an electronic device and a storage medium, aiming to improve the efficiency of problem feedback.

[0064] The problem feedback method, apparatus, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the problem feedback method in this application is described.

[0065] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0066] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0067] The problem feedback method provided in this application relates to the field of artificial intelligence technology. The problem feedback method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the problem feedback method, but is not limited to the above forms.

[0068] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0069] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards of the relevant countries and regions. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data for the proper functioning of the embodiments of this application obtained.

[0070] Figure 1 This is an optional flowchart of the problem feedback method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S104.

[0071] Step S101: In response to the first interactive event of the terminal, display the problem feedback interface corresponding to the first interactive event;

[0072] Step S102: In response to the second interactive event triggered on the problem feedback interface, obtain feedback voice information, and perform speech recognition on the feedback voice information to obtain the first text information;

[0073] Step S103: Perform semantic analysis on the first text information to obtain the semantic analysis result, call back the corresponding application interface according to the semantic analysis result, and capture the first image information by image capture of the application interface.

[0074] Step S104: Generate a problem feedback file based on the first text information and the first image information, and upload the problem feedback file to the server.

[0075] Steps S101 to S104, as illustrated in this embodiment, directly display the corresponding problem feedback interface through the terminal's interactive events, eliminating the need for users to search for feedback entry points in the application interface, thus improving the efficiency of problem feedback. Furthermore, by acquiring feedback voice information and performing voice recognition through interactive events triggered on the problem feedback interface, users are not required to input questions, saving operation time and further improving the efficiency of problem feedback. Finally, by performing semantic analysis on the recognized text information and calling back the corresponding application interface for image capture based on the semantic analysis results, image information of the application interface where the problem occurred can be automatically obtained, improving the comprehensiveness and real-time nature of problem feedback, and further enhancing the efficiency of problem feedback.

[0076] In some embodiments, the first interactive event includes at least one of the following: a shake event, a rotation event, a press event, a wrist raise event, a wrist flip event, a head down event, a head up event, a head shake event, a voice input event, and a motion-sensing event; the second interactive event includes at least one of the following: a click event, a long press event, a zoom event, a swipe event, and a drag event.

[0077] Specifically, an interaction event is an event triggered by human-computer interaction between the user and the terminal. In this embodiment of the application, the terminal executing this method can be a smartphone with a touchscreen, a smart bracelet, a smartwatch, a smart in-vehicle device, a smart TV device, or a game console, etc.

[0078] It is understandable that the first interaction event is a user's interaction operation based on the terminal level, which is used to trigger the terminal's problem feedback function and display the problem feedback interface. The first interaction event can be at least one of the following: shaking event, rotation event, pressing event, wrist raising event, wrist flipping event, head down event, head up event, head shaking event, voice input event, and motion sensing event.

[0079] Understandably, the second interactive event enables the user to trigger or terminate the terminal's voice recording function through interactive operations at the touchscreen level on the terminal. The second interactive event includes at least one of the following: click event, long press event, zoom event, swipe event, and drag event.

[0080] Please see Figure 2 In some embodiments, step S101 may include, but is not limited to, steps S1011 to S1013:

[0081] Step S1011: Determine the interaction identifier of the first interactive event;

[0082] Step S1012: Match the corresponding problem feedback interface in the preset problem feedback interface library according to the interaction identifier;

[0083] Step S1013: Display the problem feedback interface on the terminal.

[0084] In step S1011 of some embodiments, an interaction identifier corresponding to an interaction event can be preset, and each interaction identifier corresponds to a corresponding user interaction action, thereby distinguishing the user's operation.

[0085] In step S1012 of some embodiments, a problem feedback interface library can be preset, which stores different problem feedback interfaces. Each problem feedback interface corresponds to a different interaction identifier. The corresponding problem feedback interface can be determined by matching and searching in the problem feedback interface library according to the determined interaction identifier.

[0086] In step S1013 of some embodiments, the problem feedback interface displayed by the terminal has different styles depending on the interaction event. For example, for elderly users, a simple interface with bold fonts can be preset, and the interaction event corresponding to the problem feedback interface can be configured as a head shaking event or head raising event, which are easy for elderly users to perform. In this way, the problem feedback interface can be automatically displayed after the corresponding interaction event is detected.

[0087] Understandably, when users encounter problems while using an application, they can bring up a problem feedback interface by shaking their phone or performing other interactive actions, eliminating the need for users to search for feedback entry points within the application interface and improving the efficiency of problem feedback.

[0088] Please see Figure 3 In some embodiments, step S102 may include, but is not limited to, steps S1021 to S1025:

[0089] Step S1021: When the first touch operation triggered by the first area is detected, the current environment is recorded by voice through the terminal;

[0090] Step S1022: When a second touch operation triggered by the first area is detected, the terminal stops voice recording and obtains feedback voice information.

[0091] Step S1023: Input the feedback voice information into the pre-trained speech recognition model to obtain candidate text information, and display the candidate text information through the question feedback interface;

[0092] Step S1024: When a third touch operation triggered by the second area is detected, return to the step of recording voice of the current environment through the terminal;

[0093] Step S1025: When the fourth touch operation triggered by the second area is detected, the candidate text information is determined to be the first text information;

[0094] Both the first and second areas are located on the issue feedback interface.

[0095] In step S1021 of some embodiments, the first touch operation can be configured as at least one of a click event, a long press event, a zoom event, a swipe event, and a drag event. When the user performs the corresponding first touch operation in the first area, the voice recording function of the terminal can be activated.

[0096] In step S1022 of some embodiments, the second touch operation can be configured as at least one of a click event, a long press event, a zoom event, a swipe event, and a drag event. When the user performs the corresponding second touch operation in the first area, the voice recording function of the terminal can be terminated and feedback voice information can be obtained.

[0097] It is understood that the first touch operation and the second touch operation can be configured as different interaction events, such as the first touch operation being a single click event and the second touch operation being a swipe event, or the first touch operation being a long press event and the second touch operation being a long press release event, etc.; the first touch operation and the second touch operation can also be configured as the same interaction event, such as both the first touch operation and the second touch operation being single click events, where the first triggered single click event is the first touch operation, and the second triggered single click event is the second touch operation, which can also achieve control over the terminal's voice recording. It should be recognized that the above examples are only illustrative of the embodiments of this application, and the specific forms of the first touch operation and the second touch operation are not limited in the embodiments of this application.

[0098] In step S1023 of some embodiments, the speech recognition model may adopt an existing speech recognition system framework, including signal analysis, acoustic model, lexicon, language model, and search / decoding.

[0099] Understandably, after signal analysis, the feedback voice information, combined with acoustic models, language models, and dictionaries, identifies multiple most likely word sequences in the search space, thus forming candidate text information that is displayed through the question feedback interface. After viewing the candidate text information displayed on the question feedback interface, users can confirm or re-record their voice through corresponding touch operations.

[0100] In step S1024 of some embodiments, the third touch operation can be configured as at least one of a click event, a long press event, a zoom event, a swipe event, and a drag event. When the user performs the corresponding third touch operation in the second area, the voice recording function of the terminal can be restarted, and then the subsequent steps of voice recording can be executed again.

[0101] In step S1025 of some embodiments, the fourth touch operation can be configured as at least one of a click event, a long press event, a zoom event, a swipe event, and a drag event. When the user performs the corresponding fourth touch operation in the second area, the candidate text information can be confirmed and the candidate text information can be input as the first text information into subsequent steps.

[0102] It is understood that the third and fourth touch operations can be configured as different interaction events, such as the first touch operation being a single click event and the second touch operation being a swipe event, or the first touch operation being a single click event and the second touch operation being a long press event, etc.; the first and second touch operations can also be configured as the same interaction event, such as both the first and second touch operations being single click events, and two click areas are set at different positions in the second area, with the first click area marked as "re-record" and the second click area marked as "confirm". The single click event triggered in the first click area is the third touch operation, and the single click event triggered in the second click area is the fourth touch operation, which can also realize the confirmation of candidate text information. It should be recognized that the above examples are only illustrative of the embodiments of this application, and the specific forms of the first and second touch operations are not limited in the embodiments of this application.

[0103] In one specific embodiment, the user can be prompted to press and hold a corresponding interface button to record voice feedback. The user can describe the problem they encounter while using the application product via voice, and release the button after completing the description. The terminal backend recognizes the user's voice feedback. If the user's voice is recognized, it is converted into candidate text information and displayed through the feedback interface so that the user can confirm whether the text is correct. If the user's voice is not recognized, the user can be prompted to re-record through the feedback interface.

[0104] It can be recognized that the embodiments of this application obtain feedback voice information and perform voice recognition through interactive events triggered on the problem feedback interface, eliminating the need for users to input questions, saving users' operation time, and further improving the efficiency of problem feedback.

[0105] Please see Figure 4 In some embodiments, step S103 may include, but is not limited to, steps S1031 to S1033:

[0106] Step S1031: Input the first text information into the pre-trained semantic analysis model to obtain the semantic analysis result;

[0107] Step S1032: Obtain the current process list of the terminal, and traverse the current process list according to the semantic analysis results to determine the corresponding first application process;

[0108] Step S1033: Display the application interface of the first application process through the terminal, and capture the application interface according to the preset image capture template to obtain the first image information;

[0109] The image cropping template includes pre-set image cropping borders, occlusion area size, and occlusion area position.

[0110] In step S1031 of some embodiments, the semantic analysis model can adopt an existing semantic analysis system framework or be trained through deep learning. After performing semantic analysis on the first text information using the semantic analysis model, a semantic analysis result can be obtained. This result can, to a certain extent, specifically characterize the application product causing the user's problem. Especially when the user does not explicitly state the name of the application product, the semantic analysis result can be used to traverse and determine the corresponding application process in the terminal's current process list. For example, a semantic analysis result of "phone cannot scan QR code to pay" may correspond to a problem with the use of payment apps such as WeChat or Alipay. Similarly, a semantic analysis result of "phone cannot view bank card statements" may correspond to a problem with the use of mobile banking apps such as Ping An Pocket Bank or UnionPay QuickPass.

[0111] It's understandable that semantic analysis is a branch of artificial intelligence (AI) and a core task of natural language processing (NLP), involving multiple disciplines such as linguistics, computational linguistics, machine learning, and cognitive linguistics. Semantic analysis refers to using various methods to learn and understand the semantic content represented by a text; any understanding of language can be categorized as semantic analysis. A text is typically composed of words, sentences, and paragraphs. Depending on the linguistic unit being understood, semantic analysis can be further broken down into lexical-level semantic analysis, sentence-level semantic analysis, and discourse-level semantic analysis. Generally, lexical-level semantic analysis focuses on acquiring or distinguishing the semantics of individual words, sentence-level semantic analysis attempts to analyze the semantics expressed by the entire sentence, and discourse-level semantic analysis aims to study the internal structure of natural language text and understand the semantic relationships between text units (which can be sentences, clauses, or paragraphs). Simply put, the goal of semantic analysis is to achieve automatic semantic analysis at various linguistic units (including words, sentences, and discourse) by building effective models and systems, thereby understanding the true semantic meaning of the entire text.

[0112] In step S1032 of some embodiments, the current process list of the terminal can be viewed by calling the terminal's resource manager; after obtaining the current process list of the terminal, the corresponding application process can be determined by traversing the current process list according to the semantic analysis results; when an application process can be uniquely determined, the application process is taken as the first application process; when an application process cannot be uniquely determined, multiple determined application processes can be displayed in pages or by swiping to switch between pages at the same time, and the first application process is determined according to the user's touch operation.

[0113] In step S1033 of some embodiments, the image capture template can be pre-set according to the application interface of different application products. On the one hand, it needs to adapt to the border of the application interface to avoid capturing other application information or personal information on the terminal desktop. On the other hand, it also needs to cover some areas on the application interface to protect user privacy, such as the user avatar area, user nickname area and user balance area of ​​Alipay interface, and the user name area and user asset area of ​​Ping An Pocket Bank interface.

[0114] It can be recognized that, by performing semantic analysis on the identified text information and calling back the corresponding application interface to capture images based on the semantic analysis results, the embodiments of this application can automatically obtain image information of the application interface where the problem occurs, thereby improving the comprehensiveness and real-time nature of problem feedback and further improving the efficiency of problem feedback; by setting image capture templates, users' personal information can be protected, thereby improving the user experience.

[0115] Please see Figure 5 In some embodiments, the step of generating a problem feedback file based on the first text information and the first image information in step S104 may include, but is not limited to, steps S1041 to S1043:

[0116] Step S1041: Extract information from the first text information based on the semantic analysis results to obtain several keywords;

[0117] Step S1042: Perform OCR text recognition on the first image information, and annotate the first image information according to the text recognition results and keywords to obtain the second image information;

[0118] Step S1043: Package the first text information and the second image information to generate a problem feedback file.

[0119] In step S1041 of some embodiments, the semantic analysis result can, to a certain extent, directionally characterize the type of problem encountered by the user. For example, if the semantic analysis result is "the mobile phone cannot scan the code to pay," it indicates that the user is encountering a payment problem; or if the semantic analysis result is "the mobile phone cannot view bank card statements," it indicates that the user is encountering a statement query problem. Based on the semantic analysis result, information can be extracted from the first text information to obtain several keywords related to the problem type.

[0120] In step S1042 of some embodiments, OCR text recognition can be performed on the captured first image information to identify the text information corresponding to each image region. Then, the keywords extracted from the information are used to annotate each region of the first image information. For example, if the semantic analysis result is "mobile phone cannot scan QR code to pay" and the keyword is "QR code", then the user QR code region of the Alipay interface can be annotated (for the QR code in the region, user privacy can be protected by setting an image template or blurring). For another example, if the semantic analysis result is "mobile phone cannot view bank card bills" and the keyword is "bill query", then the account details region of the Ping An Pocket Bank interface can be annotated (for information in other regions of the interface, user privacy can be protected by setting an image template or blurring).

[0121] In step S1043 of some embodiments, the problem feedback file generated by packaging the first text information and the second image information can be uploaded to the server, and the problem can be analyzed by the server or manually and dispatched to the corresponding processing personnel.

[0122] It can be recognized that, by extracting keywords from the first text information and labeling the first image information according to the keywords, the embodiments of this application enable subsequent processing personnel to quickly understand the problems encountered by users, thereby improving the efficiency of problem feedback processing.

[0123] Please see Figure 6 In some embodiments, the step of generating a problem feedback file based on the first text information and the first image information in step S104 may include, but is not limited to, steps S1044 to S1047:

[0124] Step S1044: Extract information from the first text information based on the semantic analysis results to obtain several keywords;

[0125] Step S1045: Perform OCR text recognition on the first image information, and annotate the first image information according to the text recognition results and keywords to obtain the second image information;

[0126] Step S1046: Obtain the terminal's system log information;

[0127] Step S1047: Package the first text information, the second image information, and the system log information to generate a problem feedback file;

[0128] The system log information includes at least one of the following: user operation path, CPU utilization, memory utilization, disk space utilization, and network status.

[0129] The implementation processes of steps S1044 and S1045 are consistent with the implementation processes of steps S1041 and S1042 described above, and will not be repeated here in the embodiments of this application.

[0130] In step S1046 of some embodiments, the terminal's system log information includes at least one of user operation path, CPU utilization, memory utilization, disk space utilization, and network status. It is understood that the terminal's system log information can characterize the application's operating environment when a problem occurs, which is especially important for bugs that are difficult to reproduce. Subsequent troubleshooting personnel can combine the system log information to more accurately analyze and determine the cause of the problem.

[0131] In step S1047 of some embodiments, the problem feedback file generated by packaging the first text information, the second image information, and the system log information can be uploaded to the server, and the problem can be analyzed by the server or manually and dispatched to the corresponding processing personnel.

[0132] It can be recognized that the embodiments of this application further improve the comprehensiveness and real-time nature of problem feedback by collecting system log information, and at the same time further improve the efficiency of problem feedback.

[0133] This embodiment of the application displays a corresponding problem feedback interface in response to a first interactive event on the terminal, acquires feedback voice information in response to a second interactive event triggered on the problem feedback interface, performs speech recognition on the feedback voice information to obtain first text information, performs semantic analysis on the first text information to obtain semantic analysis results, calls back the corresponding application interface based on the semantic analysis results and performs image capture to obtain first image information, and finally generates a problem feedback file based on the first text information and the first image information and uploads it to the server. This embodiment of the application directly displays the corresponding problem feedback interface through the terminal's interactive events, eliminating the need for users to search for feedback entry points in the application interface, thus improving the efficiency of problem feedback; acquiring feedback voice information and performing speech recognition through interactive events triggered on the problem feedback interface eliminates the need for users to input questions, saving user operation time and further improving the efficiency of problem feedback; performing semantic analysis on the recognized text information and calling back the corresponding application interface based on the semantic analysis results to perform image capture can automatically acquire image information of the application interface where the problem occurred, improving the comprehensiveness and real-time nature of problem feedback, and further improving the efficiency of problem feedback.

[0134] It can be recognized that, based on the problem feedback method of this application, users only need to use simple interactive operations (such as gestures, voice, etc.) to provide feedback on problems, avoiding complex text input and image capture operations. This feedback method has certain advantages for application product compatibility issues (such as incomplete display or incorrect position on some models), because for compatibility issues, the data analyzed from the operation path and logs are normal, only the displayed application interface is abnormal.

[0135] Furthermore, based on the problem feedback method of this application, product developers no longer need to directly contact users to obtain on-site information about the problem or allow users to try to reproduce the problem, saving a lot of communication costs; at the same time, less disturbance to users also makes users more willing to provide feedback, which helps product developers improve the quality of application products and enhance the user experience.

[0136] Please see Figure 7 This application also provides a problem feedback device that can implement the above-mentioned problem method. The problem feedback device includes:

[0137] The interface display module is configured to respond to the first interactive event of the terminal and display the problem feedback interface corresponding to the first interactive event.

[0138] The speech recognition module is configured to respond to a second interactive event triggered on the question feedback interface, acquire feedback speech information, and perform speech recognition on the feedback speech information to obtain first text information;

[0139] The image capture module is configured to perform semantic analysis on the first text information to obtain the semantic analysis result, call back the corresponding application interface based on the semantic analysis result, and capture the first image information by capturing the image of the application interface.

[0140] The file upload module is configured to generate a problem feedback file based on the first text information and the first image information, and then upload the problem feedback file to the server.

[0141] The specific implementation of this problem feedback device is basically the same as the specific implementation of the problem feedback method described above, and will not be repeated here.

[0142] This application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, it implements the aforementioned problem feedback method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0143] Please see Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0144] The processor 801 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0145] The memory 802 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called and executed by the processor 801 using the problem feedback method of the embodiments of this application.

[0146] The 803 input / output interface is used to implement information input and output.

[0147] The communication interface 804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0148] Bus 805 transmits information between various components of the device (e.g., processor 801, memory 802, input / output interface 803, and communication interface 804);

[0149] The processor 801, memory 802, input / output interface 803, and communication interface 804 are connected to each other within the device via bus 805.

[0150] This application also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, which can be executed by one or more processors to implement the above-mentioned problem feedback method.

[0151] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0152] The problem feedback method, device, electronic device, and storage medium provided in this application directly display the corresponding problem feedback interface through terminal interaction events, eliminating the need for users to search for feedback entry points in the application interface, thus improving the efficiency of problem feedback. By acquiring feedback voice information and performing voice recognition through interaction events triggered on the problem feedback interface, users are not required to input questions, saving operation time and further improving the efficiency of problem feedback. Furthermore, by performing semantic analysis on the recognized text information and calling back the corresponding application interface for image capture based on the semantic analysis results, image information of the application interface where the problem occurred can be automatically obtained, improving the comprehensiveness and real-time nature of problem feedback, and further enhancing the efficiency of problem feedback.

[0153] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0154] It will be understood by those skilled in the art that Figures 1-6The technical solutions shown do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0155] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0156] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0157] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0158] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0159] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0160] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0161] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0162] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0163] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A problem feedback method, characterized in that, The method includes: In response to the first interactive event on the terminal, a problem feedback interface corresponding to the first interactive event is displayed; In response to a second interactive event triggered on the problem feedback interface, feedback voice information is obtained, and the feedback voice information is subjected to speech recognition to obtain first text information; The first text information is subjected to semantic analysis to obtain the semantic analysis result. The corresponding application interface is called back according to the semantic analysis result, and the first image information is obtained by cropping the application interface. A problem feedback file is generated based on the first text information and the first image information, and the problem feedback file is uploaded to the server; The step of performing semantic analysis on the first text information to obtain a semantic analysis result, calling back the corresponding application interface based on the semantic analysis result, and capturing an image of the application interface to obtain the first image information includes: The first text information is input into a pre-trained semantic analysis model to obtain the semantic analysis results; Obtain the current process list of the terminal, and traverse the current process list according to the semantic analysis results to determine the corresponding first application process; The terminal displays the application interface of the first application process, and the application interface is captured according to the preset image capture template to obtain the first image information. The image cropping template includes a pre-set image cropping border, occlusion area size, and occlusion area position. The first application process includes: when an application process is uniquely determined, the application process is used as the first application process; when an application process cannot be uniquely determined, multiple determined application processes are simultaneously displayed in pages or switched by swiping; and the first application process is determined based on the user's touch operation. The step of generating a problem feedback file based on the first text information and the first image information includes: Based on the semantic analysis results, information is extracted from the first text information to obtain several keywords; The first image information is subjected to OCR text recognition, and the first image information is labeled according to the text recognition result and the keywords to obtain the second image information; The first text information and the second image information are packaged and processed to generate a problem feedback file.

2. The problem feedback method according to claim 1, characterized in that, The response to the first interactive event on the terminal, displaying a problem feedback interface corresponding to the first interactive event, includes: Determine the interaction identifier of the first interaction event; Match the corresponding problem feedback interface in the preset problem feedback interface library according to the interaction identifier; The problem feedback interface is displayed on the terminal.

3. The problem feedback method according to claim 1, characterized in that, The response to the second interactive event triggered on the problem feedback interface, obtaining feedback voice information, and performing speech recognition on the feedback voice information to obtain first text information, includes: When a first touch operation triggered by the first area is detected, the terminal records voice of the current environment. When a second touch operation triggered by the first area is detected, the terminal stops voice recording and obtains the feedback voice information; The feedback voice information is input into a pre-trained speech recognition model to obtain candidate text information, which is then displayed through the question feedback interface. When a third touch operation triggered by the second area is detected, return to the step of recording voice of the current environment through the terminal; When a fourth touch operation triggered by the second area is detected, the candidate text information is determined to be the first text information; Both the first area and the second area are located on the problem feedback interface.

4. The problem feedback method according to any one of claims 1 to 3, characterized in that: The first interactive event includes at least one of the following: shaking event, rotation event, pressing event, wrist raising event, wrist flipping event, head down event, head up event, head shaking event, voice input event, and motion sensing event; The second interaction event includes at least one of the following: click event, long press event, zoom event, swipe event, and drag event.

5. A problem feedback method, characterized in that, The method includes: In response to the first interactive event on the terminal, a problem feedback interface corresponding to the first interactive event is displayed; In response to a second interactive event triggered on the problem feedback interface, feedback voice information is obtained, and the feedback voice information is subjected to speech recognition to obtain first text information; The first text information is subjected to semantic analysis to obtain the semantic analysis result. The corresponding application interface is called back according to the semantic analysis result, and the first image information is obtained by cropping the application interface. A problem feedback file is generated based on the first text information and the first image information, and the problem feedback file is uploaded to the server; The step of performing semantic analysis on the first text information to obtain a semantic analysis result, calling back the corresponding application interface based on the semantic analysis result, and capturing an image of the application interface to obtain the first image information includes: The first text information is input into a pre-trained semantic analysis model to obtain the semantic analysis results; Obtain the current process list of the terminal, and traverse the current process list according to the semantic analysis results to determine the corresponding first application process; The terminal displays the application interface of the first application process, and the application interface is captured according to the preset image capture template to obtain the first image information. The image cropping template includes a pre-set image cropping border, occlusion area size, and occlusion area position. The first application process includes: when an application process is uniquely determined, the application process is used as the first application process; when an application process cannot be uniquely determined, multiple determined application processes are simultaneously displayed in pages or switched by swiping; and the first application process is determined based on the user's touch operation. The step of generating a problem feedback file based on the first text information and the first image information includes: Based on the semantic analysis results, information is extracted from the first text information to obtain several keywords; The first image information is subjected to OCR text recognition, and the first image information is labeled according to the text recognition result and the keywords to obtain the second image information; Obtain the system log information of the terminal; The first text information, the second image information, and the system log information are packaged and processed to generate a problem feedback file; The system log information includes at least one of the following: user operation path, CPU utilization, memory utilization, disk space utilization, and network status.

6. The problem feedback method according to claim 5, characterized in that: The first interactive event includes at least one of the following: shaking event, rotation event, pressing event, wrist raising event, wrist flipping event, head down event, head up event, head shaking event, voice input event, and motion sensing event; The second interaction event includes at least one of the following: click event, long press event, zoom event, swipe event, and drag event.

7. A problem feedback device, characterized in that, The apparatus is used to implement the steps of the problem feedback method according to any one of claims 1 to 6, the apparatus comprising: The interface display module is configured to respond to the first interactive event of the terminal and display the problem feedback interface corresponding to the first interactive event. The speech recognition module is configured to respond to a second interactive event triggered on the problem feedback interface, acquire feedback speech information, and perform speech recognition on the feedback speech information to obtain first text information. The image capture module is configured to perform semantic analysis on the first text information to obtain the semantic analysis result, call back the corresponding application interface based on the semantic analysis result, and capture the first image information by performing image capture on the application interface. The file upload module is configured to generate a problem feedback file based on the first text information and the first image information, and upload the problem feedback file to the server.

8. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, it implements the steps of the problem feedback method as described in any one of claims 1 to 6.

9. A storage medium, said storage medium being a computer-readable storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the problem feedback method as described in any one of claims 1 to 6.

Images