Data question answering method, device and system based on talkback terminal

Abstract

The invention provides a data question and answer method, device and system based on an intercom terminal, and relates to the technical field of remote interaction, and the method comprises the steps: carrying out the text conversion of request voice data when the request voice data sent by the intercom terminal in an industrial and mining production environment is obtained; performing problem type identification on the to-be-queried text information obtained by text conversion by utilizing an industry classification model; and based on the identified target question type, performing data query on the to-be-queried text information by using an industry large model, performing voice conversion on queried question and answer result data, and sending obtained response voice data to the talkback terminal. In this way, the talkback terminal and the large model service system are combined for data question and answer, the data acquisition process is simplified, the working efficiency and convenience are improved, centralized management of the access end is facilitated, and the operation consistency is ensured; and meanwhile, the industry classification model is combined with the industry large model, so that more accurate question and answer result data can be obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of remote interaction, and in particular, to a data question-answering method, device, and system based on an intercom terminal. Background Art

[0002] With the rapid development of large model (referred to as large model for short) technology, its applications have gradually penetrated into all walks of life, and the industrial and mining fields are no exception. In this industry, large models are gradually being integrated into the production process and playing an increasingly important role. Currently, intelligent devices such as sensors are widely used in the production process of industrial and mining enterprises. During the on-site personnel inspection, it is necessary to immediately understand the production information to facilitate production activities such as inspection, problem troubleshooting, and problem solving. The original inspection method was to obtain relevant information by contacting the corresponding central control personnel, however, this method has the problem of poor real-time performance and is difficult to meet the information requirements of modern industrial and mining enterprises for high efficiency and accuracy.

[0003] In the existing industrial and mining production environment, the main challenges faced by front-line operations include, but are not limited to, network security restrictions and lack of technical operation convenience. On the one hand, in order to ensure the security and stability of the enterprise's internal network, there are usually strict specific requirements for access devices, which leads to many restrictions when front-line employees use conventional smart devices such as mobile phones and tablets, and they cannot smoothly access the network to carry out work. On the other hand, due to the complex and changeable production environment - such as harsh weather conditions, high noise levels, and dusty work sites, the operation of smart devices becomes extremely inconvenient, especially in the login process and information input process, seriously affecting work efficiency and accuracy.

[0004] In addition, the traditional method of relying on central control personnel to provide information not only has a slow response speed, but also is prone to communication errors or delays, and cannot support on-site decision-making in a timely and effective manner. Therefore, how to implement a safe and efficient instant information query method has become a key problem to be solved urgently. Summary of the Invention

[0005] The purpose of the present invention is to provide a data question-answering method, device, and system based on an intercom terminal to simplify the data acquisition process in the industrial and mining production environment and improve work efficiency and convenience.

[0006] In the first aspect, the present invention provides a data question-answering method based on an intercom terminal, which is applied to a large model business system, and the large model business system is communicatively connected to the intercom terminal; the method includes:

[0007] When the request voice data sent by the intercom terminal in the industrial and mining production environment is obtained, perform text conversion on the request voice data to obtain the text information to be queried;

[0008] Use the trained industry classification model to identify the problem type of the text information to be queried, and obtain the target problem type; among them, the industry classification model is trained based on the industry knowledge corresponding to the industrial and mining production environment;

[0009] According to the target problem type, use the trained industry large model to query data for the text information to be queried, and obtain the Q&A result data; among them, the industry large model is trained based on the industry knowledge Q&A samples;

[0010] Perform voice conversion on the Q&A result data to obtain the response voice data;

[0011] Send the response voice data to the intercom terminal so that the intercom terminal can perform voice broadcast on the response voice data.

[0012] In an optional implementation manner, performing text conversion on the request voice data to obtain the text information to be queried includes:

[0013] Perform voice recognition on the request voice data through a voice recognition model to obtain a voice recognition result; among them, the voice recognition model is trained based on the industry-specific terms and at least one dialect corresponding to the intercom terminal;

[0014] Determine the text information to be queried according to the voice recognition result and the production site data in the industrial and mining production environment where the intercom terminal is located.

[0015] In an optional implementation manner, determining the text information to be queried according to the voice recognition result and the production site data in the industrial and mining production environment where the intercom terminal is located includes:

[0016] Judge whether the voice recognition result is empty;

[0017] If the voice recognition result is not empty, correct the voice recognition result according to the production site data to obtain the text information to be queried.

[0018] In an optional implementation manner, the above method further includes:

[0019] If the voice recognition result is empty, generate a preset number of recommended questions according to the production site data;

[0020] Perform voice conversion on the recommended questions to obtain the guiding voice data;

[0021] Send the guiding voice data to the intercom terminal so that the intercom terminal can perform voice broadcast on the guiding voice data.

[0022] In an optional implementation manner, according to the target problem type, using the trained industry large model to query data for the text information to be queried, and obtaining the Q&A result data includes:

[0023] The historical Q&A records are associated with the text information to be queried by the industry large model, and the keywords are captured to obtain the initial keyword data;

[0024] The pre-constructed index library and the similar word graph are called by the industry large model to check, correct and expand the initial keyword data to obtain the target keyword data;

[0025] Based on the target keyword data and the target question type, the text information to be queried is rewritten by the industry large model to obtain the target question;

[0026] The Q&A result data corresponding to the target question is determined by the industry large model.

[0027] In an alternative embodiment, determining the Q&A result data corresponding to the target question by the industry large model includes:

[0028] The initial answer data is retrieved for the target question by the industry large model;

[0029] It is judged by the industry large model whether the initial answer data meets the preset answer refinement requirements; the answer refinement requirements include that the target question type belongs to the specified question type and the data volume of the initial answer data is greater than the preset data volume threshold;

[0030] If the answer refinement requirements are met, the key steps of the initial answer data are extracted by the industry large model to obtain the Q&A result data.

[0031] In an alternative embodiment, the Q&A result data is converted into response voice data, including:

[0032] The data format of the Q&A result data is converted by using regular formatting and natural language processing technologies to obtain the target text data, and the target text data conforms to the preset language habits and the production site voice broadcast requirements of the industrial and mining production environment;

[0033] The target text data is synthesized into voice by using the preset voice synthesis model to obtain the response voice data.

[0034] In a second aspect, the present invention provides a data Q&A device based on an intercom terminal, which is applied to a large model service system, and the large model service system is communicatively connected to the intercom terminal; the device includes:

[0035] The first conversion module is configured to perform text conversion on the requested voice data to obtain the text information to be queried when the requested voice data sent by the intercom terminal in the industrial and mining production environment is obtained;

[0036] A type recognition module is used to recognize the problem type of the text information to be queried by using the trained industry classification model, and obtain the target problem type. Among them, the industry classification model is trained based on the industry knowledge corresponding to the industrial and mining production environment.

[0037] A data query module is used to query the data of the text information to be queried by using the trained industry large model according to the target problem type, and obtain the Q&A result data. Among them, the industry large model is trained based on the industry knowledge Q&A samples.

[0038] A second conversion module is used to perform voice conversion on the Q&A result data to obtain the response voice data.

[0039] A data sending module is used to send the response voice data to the intercom terminal, so that the intercom terminal performs voice broadcast on the response voice data.

[0040] In a third aspect, the present invention provides a large model service system, including a memory and a processor. A computer program that can run on the processor is stored in the memory. When the processor executes the computer program, the method in any one of the foregoing embodiments is implemented.

[0041] In a fourth aspect, the present invention provides a data Q&A system based on an intercom terminal, including an intercom terminal and the large model service system of the foregoing embodiment. The large model service system is communicatively connected to the intercom terminal.

[0042] In the data Q&A method, device and system based on the intercom terminal provided by the present invention, when the large model service system obtains the request voice data sent by the intercom terminal in the industrial and mining production environment, it can perform text conversion on the request voice data to obtain the text information to be queried; use the trained industry classification model to recognize the problem type of the text information to be queried, and obtain the target problem type. Among them, the industry classification model is trained based on the industry knowledge corresponding to the industrial and mining production environment; according to the target problem type, use the trained industry large model to query the data of the text information to be queried, and obtain the Q&A result data. Among them, the industry large model is trained based on the industry knowledge Q&A samples; perform voice conversion on the Q&A result data to obtain the response voice data; send the response voice data to the intercom terminal, so that the intercom terminal performs voice broadcast on the response voice data. In this way, the intercom terminal and the large model service system are combined for data Q&A. Users only need to use the intercom terminal to realize the Q&A interaction of the whole-process voice, realize efficient and convenient data Q&A, can greatly simplify the data acquisition process in the industrial and mining production environment, significantly improve the work efficiency and convenience, and is also convenient for centralized management of the access end, and can ensure the consistency and simplicity of operations; and, combining the industry classification model and the industry large model can obtain more accurate Q&A result data and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0044] Figure 1 It is a schematic flowchart of a data question-answering method based on an intercom terminal provided by an embodiment of the present invention;

[0045] Figure 2 It is a schematic structural diagram of a data question-answering device based on an intercom terminal provided by an embodiment of the present invention;

[0046] Figure 3 It is a schematic structural diagram of a large model business system provided by an embodiment of the present invention;

[0047] Figure 4 It is a schematic structural diagram of a data question-answering system based on an intercom terminal provided by an embodiment of the present invention;

[0048] Figure 5 It is a schematic interaction flowchart of a data question-answering system based on an intercom terminal provided by an embodiment of the present invention. Specific embodiments

[0049] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments. Obviously, the described embodiments are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0050] In today's industrial and mining fields, the production environment generally faces many challenges, which have a significant impact on the use of mobile intelligent devices in on-site production operations. Specifically, in the actual use of intelligent devices such as mobile phones and tablets, a series of obstacles have been encountered. On the one hand, network security restrictions have become a key factor restricting the use of devices. Due to strict specific requirements for accessing devices, frontline employees are often restricted when using conventional mobile phones, tablets and other devices and cannot smoothly access the network to carry out work. On the other hand, the convenience of operating intelligent devices is also relatively prominent. In the login process and information input process, due to the complexity of the production environment, such as harsh climate conditions, dusty and noisy work sites, etc., employees are difficult to operate efficiently, and the input process is extremely inconvenient, seriously affecting work efficiency and accuracy.

[0051] Based on this, a data question-and-answer method, device, and system based on an intercom terminal provided by an embodiment of the present invention will provide a brand-new way for customers to easily obtain production data by simply operating an intercom terminal such as a walkie-talkie, greatly simplifying the data acquisition process and significantly improving work efficiency and convenience.

[0052] To facilitate the understanding of this embodiment, a data question-and-answer method based on an intercom terminal disclosed in an embodiment of the present invention will be introduced in detail first.

[0053] An embodiment of the present invention provides a data question-and-answer method based on an intercom terminal. This method is applied to a large model business system, and the large model business system is communicatively connected to an authorized intercom terminal; among them, the intercom terminal can be a device such as a walkie-talkie or a smart watch that supports voice data sending and receiving. Refer to Figure 1 The flowchart of a data question-and-answer method based on an intercom terminal shown. This method mainly includes the following steps S110 to step S150:

[0054] Step S110, when the requested voice data sent by the intercom terminal in the industrial and mining production environment is obtained, perform text conversion on the requested voice data to obtain the text information to be queried.

[0055] Users usually use the intercom terminal to perform voice question-and-answer operations in the industrial and mining production environment; the intercom terminal can directly communicate with the large model business system through tools such as gateways to complete the question-and-answer operation; the intercom terminal can also perform data transmission with the large model business system through a dispatching device to complete the question-and-answer operation. Here, the voice data transmitted by the user to the large model business system through the intercom terminal is called the requested voice data. After receiving the requested voice data, the large model business system needs to first convert the requested voice data into text to obtain the text information to be queried. The requested voice data can be subjected to speech recognition through a speech recognition model, and the recognized speech recognition result is used as the text information to be queried.

[0056] In some possible embodiments, to improve the accuracy of speech recognition, the step of performing text conversion on the requested voice data to obtain the text information to be queried may include: performing speech recognition on the requested voice data through a speech recognition model to obtain a speech recognition result; among them, the speech recognition model is trained based on industry-specific terms corresponding to the intercom terminal and at least one dialect; determining the text information to be queried according to the speech recognition result and the production site data in the industrial and mining production environment where the intercom terminal is located...

[0057] The above at least one dialect may include the dialect of the production place where the intercom terminal is located, and may also include dialects of other regions. The speech recognition model has high adaptability and intelligence. It can not only accurately recognize industry-specific terms, but also effectively process dialects of various regions, fully considering the language habits and work scenario requirements of different users.

[0058] Optionally, the step of determining the text information to be queried according to the speech recognition result and the production site data in the industrial and mining production environment where the intercom terminal is located may include: determining whether the speech recognition result is empty; if the speech recognition result is not empty, correcting the speech recognition result according to the production site data to obtain the text information to be queried. Wherein, the text information to be queried is the corrected speech recognition result.

[0059] The above production site data may be real-time monitoring data obtained in real time by the large model business system through Internet of Things devices such as sensors arranged at the production site, or historical monitoring data of the production site pre-stored in the large model business system. The large model business system supports intelligent correction of the speech recognition result in combination with the actual situation of the production site. For example, taking a coal mine production site as an example, when the user's voice input is "coal quantity of belt 501", the large model business system can accurately correct it to "instantaneous coal quantity of belt 501_1" based on the production site data. In this embodiment, the correction rule has high flexibility and can be designed standardly or customized based on a powerful large language model according to the specific requirements of the user. This customization ability ensures that the large model business system can fit the business processes and data specifications of different users and meet diverse actual needs.

[0060] Optionally, feature extraction may be performed on the speech recognition result and the production site data to obtain target input features, the target input features are input into a pre-trained correction model, and the output of the correction model is used as the text information to be queried; wherein, the correction model can predict the correct text corresponding to the speech recognition result.

[0061] The training process of the above correction model may be as follows: collect historical speech data and its corresponding production site data and correct text labels, perform speech recognition on the historical speech data to obtain historical text data, perform feature extraction on the historical text data and its production site data to obtain sample input data; train a correction model according to the sample input data and its correct text labels using machine learning or deep learning algorithms (such as hidden Markov model, long short-term memory network, etc.).

[0062] Further, the above method may further include: if the speech recognition result is empty, generating a preset number of recommended questions based on the production site data; performing speech conversion on the recommended questions to obtain guiding speech data; and sending the guiding speech data to the intercom terminal so that the intercom terminal can perform voice broadcast on the guiding speech data.

[0063] The above preset number can be set according to actual needs and is not limited here. For example, the preset number is 3. In this embodiment, when the content recognized by speech is unclear and cannot be accurately judged, the large model business system will combine the rich data and knowledge of the production site to intelligently generate three recommended questions. These recommended questions will be clearly broadcast through voice on the intercom terminal to guide users to perform correct question-and-answer operations, help users express their needs more accurately, and improve the success rate and efficiency of data query.

[0064] Optionally, the step of generating a preset number of recommended questions based on the production site data may include: preprocessing the production site data (such as data cleaning and format conversion, etc.) to obtain the preprocessed production site data; inputting the preprocessed production site data into a large language model to obtain a preset number of recommended questions output by the large language model. Among them, the large language model pre-stores preset question types and the number of recommended questions, and also stores production process information, key performance indicators, the main challenges or bottlenecks currently faced, and specific questions that are expected to be solved through questioning, etc. Therefore, the large language model can analyze the production site data and output the required number of recommended questions corresponding to the production site data.

[0065] Step S120, using the trained industry classification model to identify the question type of the text information to be queried, and obtaining the target question type; wherein, the industry classification model is trained based on the industry knowledge corresponding to the industrial and mining production environment.

[0066] In this embodiment, multiple question types are pre-divided, such as knowledge Q&A, data, report, and system operation, etc., and a classifier is trained using industry knowledge data and corresponding question type labels to obtain an industry classification model capable of identifying these question types. Inputting the text information to be queried into the industry classification model, the industry classification model can output the target question type, and the target question type can be one of knowledge Q&A, data, report, and system operation, etc. Compared with the general classifier, the accuracy of the industry classification model trained with industry knowledge has been greatly improved, and the misassignment of question types can be avoided.

[0067] Step S130, according to the target question type, using the trained industry large model to query data for the text information to be queried, and obtaining the Q&A result data; wherein, the industry large model is trained based on industry knowledge Q&A samples.

[0068] In this embodiment, an innovative classification architecture that coordinates the industry large model and the industry classification model is proposed, achieving domain adaptation of intent recognition through combination with the industry classification model.

[0069] After being processed by the speech recognition model, the voice sent by the intercom terminal is successfully converted into text information. This text will be quickly pushed to the large model data query platform. The large model data query platform stores the trained industry large model and thus has powerful data analysis and query capabilities, and can accurately query the data corresponding to its problem type according to the input text, covering various types such as data query and knowledge query. After the large model data query platform queries the relevant data, it returns the data to the large model business system in a timely manner, so that the large model business system can obtain the question and answer result data.

[0070] In some possible embodiments, the above step S130 may include: associating the historical question and answer records of the text information to be queried through the industry large model and capturing keywords to obtain initial keyword data; calling the pre-constructed index library and the similar word graph through the industry large model to check, correct and expand the initial keyword data to obtain target keyword data; rewriting the text information to be queried through the industry large model based on the target keyword data and the target question type to obtain a target question; determining the question and answer result data corresponding to the target question through the industry large model.

[0071] After obtaining the text information to be queried, the industry large model can automatically capture keywords and accurately extract elements such as device information and attribute information from the user's colloquial questions (such as: What is the current current of xxx?). To obtain a more accurate target question, this embodiment provides a function of remembering while chatting. Each intercom terminal has an identifier (such as a number), and the large model business system can record the question and answer results of each intercom terminal. Before retrieving the answer to the current target question, it searches the historical question and answer records of the intercom terminal and determines the historical question and answer records related to the current target question, which is called the associated question and answer record here. Thus, when capturing the keywords of the text information to be queried, the associated question and answer records are also considered to obtain more comprehensive initial keyword data. This function of remembering while chatting can support situations such as interrupting in the middle and asking follow-up questions (such as first asking "Today's output" and then asking "How about compared with last week").

[0072] Afterwards, the industry large model can check the metric definitions in the initial keyword data based on the metric library to avoid confusing substances such as "flocculant" and "coagulant". When there are inaccurate metrics in the initial keyword data, they can be automatically corrected, or recommended metrics can be generated and returned to the user for confirmation, that is, corresponding voice is generated and sent to the intercom terminal for playback, such as "Please confirm whether AA should be BB". When there are no inaccurate metrics in the initial keyword data, no correction is required. After checking and correction, automatic keyword expansion can be performed based on the similar word graph before searching to obtain the target keyword data. For example, when the user asks "What should I do if the equipment overheats", synonyms such as "high temperature" and "heat dissipation" will be searched simultaneously. These processing procedures can improve the accuracy of the subsequent obtained target questions, thereby ensuring that users can obtain accurate answers.

[0073] When rewriting the question of the text information to be queried, the target keyword data can be filled into the question sentence pattern corresponding to the target question type to obtain the initial question. Determine whether the initial question belongs to a complex question. For example, determine whether there are multiple devices and / or multiple metrics in the initial question. If so, the initial question belongs to a complex question; otherwise, the initial question does not belong to a complex question. If it belongs to a complex question, disassemble the initial question to obtain the target questions, and at this time, there are multiple target questions. If it does not belong to a complex question, directly determine the initial question as the target question, and at this time, there is one target question. This embodiment can disassemble complex questions into small questions. For example, "The equipment alarms E05" can be disassembled into the meaning of the fault code, the processing process, and precautions. Another example is that "What are the currents of equipment A and B" can be disassembled into what is the current of equipment A and what is the current of equipment B. Through question rewriting, it is possible to avoid missing some questions and ensure that users can obtain all the answers they need.

[0074] Optionally, the steps of determining the Q&A result data corresponding to the target question by the industry large model may include: retrieving answers to the target question through the industry large model to obtain the initial answer data; determining whether the initial answer data meets the preset answer refinement requirements through the industry large model; the answer refinement requirements include that the target question type belongs to the specified question type and the data volume of the initial answer data is greater than the preset data volume threshold; if the answer refinement requirements are met, key steps are extracted from the initial answer data through the industry large model to obtain the Q&A result data.

[0075] The above specified question type can be set according to actual needs and is not limited here. For example, the specified question type is knowledge Q&A. This embodiment provides an answer refinement technique, which can refine answers with more content such as knowledge Q&A and output the key information required by users. For example, key steps are automatically extracted from 10 pages of instructions to generate a "Three-step Emergency Operation Guide".

[0076] Furthermore, before searching for answers to the target questions, it is also possible to determine whether the target questions are in violation of regulations based on the preset violation corpus, and refuse to answer the target questions that are in violation of regulations. This approach improves data security.

[0077] Furthermore, when an answer to the target question is retrieved, a friendly reminder that no relevant information was found can be given, rather than a fabricated answer. A recommended answer can also be provided, with the explanation that it was generated based on historical or relevant data. This approach allows users to understand the credibility of the answer and choose to adopt it.

[0078] Step S140: Perform voice conversion on the question and answer result data to obtain answer voice data.

[0079] In some possible embodiments, in order to improve the readability and usability of the data, the step of performing voice conversion on the question and answer result data to obtain response voice data may include: using regular formatting and natural language processing technology to convert the data format of the question and answer result data to obtain target text data, which conforms to the preset language habits and production site voice broadcast requirements of the industrial and mining production environment; using a preset voice synthesis model to perform voice synthesis on the target text data to obtain response voice data.

[0080] Regular formatting refers to the use of regular expressions to perform text search, matching, and replacement operations to achieve the purpose of text formatting. The large-scale business system meticulously processes the question-and-answer result data, applying advanced technologies such as regular formatting and natural language processing to automatically convert the business data into colloquial explanations (for example, "The current of xxx device is 1.4A" rather than the business result). This makes the data format more consistent with everyday language habits and the expression styles used in production sites. Optionally, standard sentence patterns corresponding to the question type are preset. The standard sentence patterns can be examples containing prompt words. When converting the data format of the question-and-answer result data, the question-and-answer result data can be filled with the standard sentence pattern corresponding to the target question type to obtain the target text data. This process not only improves the readability and usability of the data, but also ensures that users can easily understand and use the query results, further enhancing the user experience. After the data format conversion, the target text data is transmitted to the speech synthesis model. This speech synthesis model uses advanced speech synthesis technology to convert the data into natural and fluent speech, generating voice information corresponding to the question-and-answer result data (i.e., the response voice data, which is consistent with the target text data).

[0081] Step S150: sending the response voice data to the intercom terminal so that the intercom terminal can perform voice broadcasting on the response voice data.

[0082] The generated response voice data can be pushed back to the dispatching device through a carefully designed protocol interface. The dispatching device is responsible for accurately transmitting the response voice data to the intercom terminal to achieve the voice broadcast function. Users only need to simply operate the intercom terminal to easily obtain clear and accurate voice answers without manually checking text information, greatly improving work efficiency and convenience.

[0083] Furthermore, in order to enhance the interaction effect, the above method further includes: after obtaining the request voice data sent by the intercom terminal, circularly sending a preset playback signal to the intercom terminal to enable the intercom terminal to circularly play a preset voice.

[0084] In order to enhance the interaction effect, after the large model business system receives the request voice data, it will synchronously and circularly push preset voices such as "Querying, please wait" to improve the interaction experience. After generating the response voice data, it will terminate the circular sending of the preset playback signal to enable the intercom terminal to terminate the circular broadcast of the preset voice and start playing the response voice data.

[0085] In the data question-and-answer method based on the intercom terminal provided by the embodiments of the present invention, when the large model business system obtains the request voice data sent by the intercom terminal in the industrial and mining production environment, it can perform text conversion on the request voice data to obtain the text information to be queried; use the trained industry classification model to identify the question type of the text information to be queried to obtain the target question type; wherein, the industry classification model is trained based on the industry knowledge corresponding to the industrial and mining production environment; according to the target question type, use the trained industry large model to query the text information to be queried to obtain the question-and-answer result data; wherein, the industry large model is trained based on the industry knowledge question-and-answer samples; perform voice conversion on the question-and-answer result data to obtain the response voice data; send the response voice data to the intercom terminal to enable the intercom terminal to perform voice broadcast on the response voice data. In this way, by combining the intercom terminal with the large model business system for data question-and-answer, users only need to use the intercom terminal to achieve the full-process voice question-and-answer interaction, realizing efficient and convenient data question-and-answer, which can greatly simplify the data acquisition process in the industrial and mining production environment, significantly improve work efficiency and convenience, and is also convenient for centralized management of the access end, as well as ensuring the consistency and simplicity of operations; moreover, by combining the industry classification model with the industry large model, more accurate question-and-answer result data can be obtained, enhancing the user experience.

[0086] Corresponding to the above data question-and-answer method based on the intercom terminal, the embodiments of the present invention also provide a data question-and-answer device based on the intercom terminal. This device is applied to the large model business system, and the large model business system is communicatively connected to the authorized intercom terminal. Refer to Figure 2 The structural schematic diagram of a data question-and-answer device based on the intercom terminal shown, this device includes:

[0087] The first conversion module 201 is configured to perform text conversion on the requested voice data when the requested voice data sent by the intercom terminal in the industrial and mining production environment is obtained, so as to obtain the text information to be queried;

[0088] The type recognition module 202 is configured to use the trained industry classification model to recognize the problem type of the text information to be queried, so as to obtain the target problem type; wherein, the industry classification model is trained based on the industry knowledge corresponding to the industrial and mining production environment;

[0089] The data query module 203 is configured to use the trained industry large model to query data for the text information to be queried, so as to obtain the Q&A result data; wherein, the industry large model is trained based on the industry knowledge Q&A samples;

[0090] The second conversion module 204 is configured to perform voice conversion on the Q&A result data to obtain the response voice data;

[0091] The data sending module 205 is configured to send the response voice data to the intercom terminal, so that the intercom terminal performs voice broadcast on the response voice data.

[0092] In the data Q&A device based on the intercom terminal provided by the embodiment of the present invention, when the large model service system obtains the requested voice data sent by the intercom terminal in the industrial and mining production environment, it can perform text conversion on the requested voice data to obtain the text information to be queried; use the trained industry classification model to recognize the problem type of the text information to be queried, so as to obtain the target problem type; wherein, the industry classification model is trained based on the industry knowledge corresponding to the industrial and mining production environment; according to the target problem type, use the trained industry large model to query data for the text information to be queried, so as to obtain the Q&A result data; wherein, the industry large model is trained based on the industry knowledge Q&A samples; perform voice conversion on the Q&A result data to obtain the response voice data; send the response voice data to the intercom terminal, so that the intercom terminal performs voice broadcast on the response voice data. In this way, the intercom terminal and the large model service system are combined for data Q&A. Users only need to use the intercom terminal to realize the Q&A interaction of the whole-process voice, realize efficient and convenient data Q&A, can greatly simplify the data acquisition process in the industrial and mining production environment, significantly improve the work efficiency and convenience, and is also convenient for centralized management of the access end, and can ensure the consistency and simplicity of operations; moreover, combining the industry classification model with the industry large model can obtain more accurate Q&A result data and improve the user experience.

[0093] Further, the first conversion module 201 is specifically configured to: perform speech recognition on the requested speech data through a speech recognition model to obtain a speech recognition result; wherein, the speech recognition model is trained based on industry-specific terms corresponding to the intercom terminal and at least one dialect; determine the text information to be queried according to the speech recognition result and the production site data in the industrial and mining production environment where the intercom terminal is located.

[0094] Further, the first conversion module 201 is further configured to: determine whether the speech recognition result is empty; if the speech recognition result is not empty, correct the speech recognition result according to the production site data to obtain the text information to be queried.

[0095] Further, the first conversion module 201 is further configured to: if the speech recognition result is empty, generate a preset number of recommended questions according to the production site data;

[0096] The second conversion module 204 is further configured to: perform speech conversion on the recommended questions to obtain guiding speech data;

[0097] The data sending module 205 is further configured to: send the guiding speech data to the intercom terminal so that the intercom terminal performs voice broadcast on the guiding speech data.

[0098] Further, the data query module 203 is specifically configured to: associate historical Q&A records and extract keywords from the text information to be queried through an industry large model to obtain initial keyword data; call a pre-constructed index library and a similar word graph through the industry large model to check, correct, and expand the initial keyword data to obtain target keyword data; rewrite the text information to be queried through the industry large model based on the target keyword data and the target question type to obtain a target question; determine the Q&A result data corresponding to the target question through the industry large model.

[0099] Further, the data query module 203 is further configured to: retrieve answers for the target question through the industry large model to obtain initial answer data; determine whether the initial answer data meets the preset answer refinement requirements through the industry large model; the answer refinement requirements include that the target question type belongs to a specified question type and the data volume of the initial answer data is greater than a preset data volume threshold; if the answer refinement requirements are met, extract key steps from the initial answer data through the industry large model to obtain the Q&A result data.

[0100] Further, the above-mentioned second conversion module 204 is specifically configured to: use regular formatting and natural language processing technologies to perform data format conversion on the Q&A result data to obtain target text data, where the target text data conforms to the preset language habits and the production site voice broadcast requirements of the industrial and mining production environment; use a preset voice synthesis model to perform voice synthesis on the target text data to obtain response voice data.

[0101] Further, the above-mentioned data sending module 205 is further configured to: after obtaining the requested voice data sent by the intercom terminal, circularly send a preset playback signal to the intercom terminal, so that the intercom terminal circularly plays the preset voice.

[0102] The data Q&A device based on the intercom terminal provided in this embodiment has the same implementation principle and the same technical effects as the foregoing data Q&A method embodiment based on the intercom terminal. For the sake of brief description, for the parts not mentioned in the data Q&A device embodiment based on the intercom terminal, reference may be made to the corresponding content in the foregoing data Q&A method embodiment based on the intercom terminal.

[0103] As Figure 3 shown, a large model service system provided by an embodiment of the present invention includes: a processor 301, a memory 302, and a bus. The memory 302 stores a computer program that can run on the processor 301. When the large model service system runs, the processor 301 communicates with the memory 302 through the bus, and the processor 301 executes the computer program to implement the above-mentioned data Q&A method based on the intercom terminal.

[0104] Specifically, the above-mentioned memory 302 and processor 301 can be general-purpose memory and processor, and no specific limitation is made here.

[0105] An embodiment of the present invention further provides a data Q&A system based on an intercom terminal. Refer to Figure 4 the structural schematic diagram of a data Q&A system based on an intercom terminal shown. The data Q&A system based on the intercom terminal includes an intercom terminal 401 and the above-mentioned large model service system 402, and the large model service system 402 is communicatively connected to the intercom terminal 401.

[0106] The above-mentioned intercom terminal 401 can be one or more, that is, the large model service system 402 can communicate with one or more intercom terminals 401 at the same time. Figure 4 Three intercom terminals 401 are shown in

[0107] In a possible implementation manner, the above-mentioned intercom terminal 401 can directly communicate with the large model service system 402 through tools such as gateways to complete the Q&A operation.

[0108] In another possible implementation, as Figure 4 shown, the above data Q&A system based on the intercom terminal further includes a dispatching device 403. The intercom terminal 401 performs data transmission with the large model business system 402 through the dispatching device 403 to complete the Q&A operation.

[0109] For ease of understanding, the following takes the intercom terminal 401 as an example of a walkie-talkie to introduce the above data Q&A system based on the intercom terminal in detail.

[0110] The objective of the embodiment of the present invention is to create an innovative platform integrating speech recognition technology and a large model Q&A system, which is particularly suitable for walkie-talkie users. Through this platform, users can ask questions in the form of voice input and immediately obtain knowledge-based or index-based answers based on the large model. This data Q&A system particularly strengthens the recognition ability of specific industry terms to ensure accurate understanding of users' query requirements in noisy or complex environments. By using the existing advanced large model Q&A technology, it not only improves the convenience and accuracy of data acquisition but also greatly enhances the work efficiency of users. In short, the embodiment of the present invention aims to provide users with an efficient, accurate, and easy-to-use professional Q&A solution, especially suitable for professionals relying on walkie-talkie communication.

[0111] The embodiment of the present invention focuses on constructing an innovative interactive system, aiming to achieve efficient and convenient data Q&A with the industry large model through the walkie-talkie and integrate the voice broadcast function. This system will provide customers with a new way to easily obtain the production data required by users by simply operating the walkie-talkie, greatly simplifying the data acquisition process and significantly improving work efficiency and convenience.

[0112] The working process of the above data Q&A system based on the intercom terminal mainly includes the following four parts:

[0113] 1) Walkie-talkie docking

[0114] The walkie-talkie establishes a close connection with the dispatching device to achieve efficient transmission of voice information. Two transmission modes are supported: the streaming mode and the blocking mode. In the streaming mode, the voice information can be pushed to the large model business system in real time and continuously to ensure the immediacy of the information; the blocking mode is applicable to scenarios with higher requirements for data integrity to ensure that the voice information reaches the large model business system completely and correctly.

[0115] To cope with the complex network environment, the large model business system has a powerful resume interrupted transfer function. When the network experiences a temporary interruption, the scheduling device does not need to resend all the voice data. Once the network resumes, the scheduling device pushes the voice data again, and the large model business system can still accurately continue to receive it, seamlessly connecting to the data transfer at the interruption point. This feature significantly improves the stability and availability of the large model business system, effectively reducing the risk of data loss or transmission interruption caused by network problems, and ensuring that the large model business system can operate continuously and stably under various network conditions.

[0116] The walkie-talkie uses the modified underlying protocol (RTSP communication protocol) to connect to the large model business system, which can achieve the effect of smooth speech. For example, from pressing the walkie-talkie to hearing the answer, the whole process takes less than 1 second (the traditional system takes more than 3 seconds).

[0117] 2) Connect to the speech recognition model

[0118] After the large model business system successfully receives the voice data transmitted by the walkie-talkie, it immediately enables an advanced speech recognition model to accurately recognize the voice input by the user. This speech recognition model has high adaptability and intelligence. It can not only accurately recognize industry-specific terms but also effectively handle dialects from various regions, fully considering the language habits and work scenario requirements of different users.

[0119] At the same time, the large model business system supports intelligent correction of the speech recognition results in combination with the actual situation of the production site. For example, taking the coal mine production site as an example, when the user's voice input is "501 coal carrying amount", the large model business system can accurately correct it to "501_1 instantaneous coal carrying amount" based on the production site data. The correction rules are highly flexible and can be designed standardly or customized based on the powerful large language model according to the specific requirements of the user. This customization ability ensures that the large model business system can fit different users' business processes and data specifications, meeting diverse actual needs.

[0120] In addition, when the recognized speech content is unclear and cannot be accurately judged, the large model business system will combine the rich data and knowledge of the production site to intelligently generate three recommended questions. These recommended questions will be clearly broadcast through the walkie-talkie by voice, guiding the user to perform correct question-and-answer operations, helping the user express their needs more accurately, and improving the success rate and efficiency of data query.

[0121] 3) Connect to the industry large model

[0122] After being processed by the speech recognition model, the voice sent by the walkie-talkie is successfully converted into text information. This text will be quickly pushed to the large model data query platform. The large model data query platform stores the trained industry large model, so it has powerful data analysis and query capabilities, and can accurately query the corresponding classified data according to the input text, covering various types such as data query and knowledge query.

[0123] After the large model data query platform queries the relevant data, it returns the data to the large model business system in a timely manner, so that the large model business system can obtain the question-and-answer result data. Subsequently, the large model business system performs fine processing on the question-and-answer result data, using advanced technologies such as regular formatting and natural language processing to make the data format more in line with the daily language habits and the daily expression formats in the production site. This processing process not only improves the readability and usability of the data, but also ensures that users can easily understand and use the query results, further enhancing the user experience.

[0124] 4) Return of speech synthesis result

[0125] The target text data obtained through data format conversion will be transmitted to the speech synthesis model. The speech synthesis model uses advanced speech synthesis technology to convert the data into natural and fluent speech, generating speech information corresponding to the question-and-answer result data.

[0126] The generated response speech data can be pushed back to the dispatching device through a carefully designed protocol interface. The dispatching device is responsible for accurately transmitting the response speech data to the walkie-talkie to achieve the voice broadcast function. Users only need to operate the walkie-talkie simply to easily obtain clear and accurate voice answers without manually viewing text information, greatly improving work efficiency and convenience.

[0127] In order to improve the interaction effect, after the large model business system receives the request voice data, it will synchronously and circularly push preset voices such as "Please wait while querying" to enhance the interaction experience. After generating the response voice data, it will terminate the circular sending of the preset playback signal, so that the intercom terminal terminates the circular broadcast of the preset voice and starts playing the response voice data.

[0128] In addition, the walkie-talkie and the large model business system adopt an advanced processing mechanism that combines business synchronization and asynchrony, effectively improving the overall processing effect of the data Q&A system based on the walkie-talkie terminal. The entire processing process is currently divided into four stages: data reception, voice conversion, industry large model Q&A, and result voice synthesis and push to the walkie-talkie. During the processing of the same request, each stage is carried out sequentially in a synchronous serial manner to ensure the processing accuracy and stability of each link; while different requests adopt a parallel processing method in the same stage, making full use of the resources of the large model business system to improve the concurrent processing ability of the large model business system. This design pattern can greatly improve the access ability of the large model business system while ensuring the quality of data processing, meet the needs of large-scale user concurrent access, and ensure that the large model business system can still operate efficiently and stably under high load.

[0129] See Figure 5 The schematic diagram of the interaction process of a data Q&A system based on a walkie-talkie terminal shown in the figure. The interaction process is as follows: The user sends voice through the walkie-talkie, and the walkie-talkie dispatching platform (i.e., the above-mentioned dispatching device) pushes the voice data to the walkie-talkie large model business system (i.e., the above-mentioned large model business system); the walkie-talkie large model business system performs voice data reception, voice recognition, recognition result processing (such as correction), large model Q&A service, Q&A result processing (such as data format conversion), and voice synthesis, and finally sends it to the walkie-talkie through the walkie-talkie dispatching platform for voice broadcast, so that the walkie-talkie broadcasts the answer; in addition, after receiving the voice data, the walkie-talkie large model business system sends a loop playback signal, that is, it loops and pushes the waiting process audio after receiving the voice. The waiting process audio is sent to the walkie-talkie through the walkie-talkie dispatching platform for voice broadcast, so that the walkie-talkie loops and broadcasts the process prompt.

[0130] In summary, the data Q&A system based on the walkie-talkie terminal provided in this embodiment uses the walkie-talkie for Q&A interaction, adopts full-process voice input and voice broadcast, supports the recognition of industry-specific terms and the recognition of production local dialect voices, supports voice-guided Q&A, and clarifies fuzzy questions through voice broadcast.

[0131] This embodiment has the following advantages:

[0132] 1. Efficient information acquisition: On-site staff only need to input voice to quickly query the required information and can receive answers through voice broadcast without manual operation. This non-contact interaction method not only improves work efficiency but also facilitates centralized management of the access end, ensuring the consistency and simplicity of operations.

[0133] 2. Professional Term and Dialect Support: The large model business system has been specially optimized for the recognition of specific industry terms and local dialects. Even in noisy or linguistically diverse environments, it can accurately understand and respond to users' questions, ensuring smooth communication.

[0134] 3. Comprehensive Information Coverage: The large model business system can provide diverse Q&A services including but not limited to real-time metric monitoring, workflow guidance, safety regulations, and production-related knowledge, covering almost all information needs in industrial and mining production and providing users with one-stop information services.

[0135] 4. Simple Operation Experience: The entire interaction process is simplified to voice input and output, greatly reducing manual intervention, adapting to complex and changing working environments, and enhancing the operation convenience and response speed of users.

[0136] 5. Enhanced Data Security: A strict data access control mechanism is incorporated into the design of the data Q&A system based on the intercom terminal. Only authorized intercom terminals can query data, effectively ensuring the security and privacy of internal information and preventing the leakage of sensitive information.

[0137] The embodiment of the present invention also provides a computer-readable storage medium. A computer program is stored on this computer-readable storage medium, and when the computer program is run by a processor, it executes the data Q&A method based on the intercom terminal in the foregoing method embodiment. The computer-readable storage medium includes various media such as USB flash drives, external hard drives, read-only memory (ROM), RAM, magnetic disks, or optical discs that can store program codes.

[0138] In all the examples shown and described here, any specific value should be construed as merely exemplary and not as a limitation. Therefore, other examples of the exemplary embodiments may have different values.

[0139] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0140] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings, direct couplings, or communication connections shown or discussed between each other can be through some communication interfaces. The indirect couplings or communication connections of devices or modules can be in electrical, mechanical, or other forms.

[0141] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] In addition, in various embodiments of the present invention, the functional modules can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data question-and-answer method based on an intercom terminal, characterized in that, Applied to the large model business system, the large model business system is communicatively connected to the intercom terminal; the method includes: When the request voice data sent by the intercom terminal in the industrial and mining production environment is obtained, perform text conversion on the request voice data to obtain the text information to be queried; Use the trained industry classification model to identify the problem type of the text information to be queried, and obtain the target problem type; wherein, the industry classification model is trained based on the industry knowledge corresponding to the industrial and mining production environment; According to the target problem type, use the trained industry large model to query the text information to be queried, and obtain the Q&A result data; wherein, the industry large model is trained based on the industry knowledge Q&A samples; Perform voice conversion on the Q&A result data to obtain the response voice data; Send the response voice data to the intercom terminal, so that the intercom terminal performs voice broadcast on the response voice data.

2. The method according to claim 1, wherein The performing text conversion on the request voice data to obtain the text information to be queried includes: Perform voice recognition on the request voice data through a voice recognition model to obtain a voice recognition result; wherein, the voice recognition model is trained based on the industry-specific terms and at least one dialect corresponding to the intercom terminal; Determine the text information to be queried according to the voice recognition result and the production site data in the industrial and mining production environment where the intercom terminal is located.

3. The method according to claim 2, characterized in that, The determining the text information to be queried according to the voice recognition result and the production site data in the industrial and mining production environment where the intercom terminal is located includes: Judge whether the voice recognition result is empty; If the voice recognition result is not empty, correct the voice recognition result according to the production site data to obtain the text information to be queried.

4. The method according to claim 3, wherein The method further includes: If the voice recognition result is empty, generate a preset number of recommended questions according to the production site data; Perform voice conversion on the recommended questions to obtain guiding voice data; Send the guiding voice data to the intercom terminal, so that the intercom terminal performs voice broadcast on the guiding voice data.

5. The method according to claim 1, characterized in that The using the trained industry large model to query the text information to be queried according to the target problem type to obtain the Q&A result data includes: Associate the historical Q&A records of the text information to be queried through the industry large model and capture keywords to obtain initial keyword data; Call the pre-constructed index library and similar word graph through the industry large model to check, correct and expand the initial keyword data to obtain target keyword data; Based on the target keyword data and the target problem type, rewrite the text information to be queried through the industry large model to obtain a target problem; Determine the Q&A result data corresponding to the target problem through the industry large model.

6. The method according to claim 5, wherein The determining the Q&A result data corresponding to the target problem through the industry large model includes: Retrieve answers to the target problem through the industry large model to obtain initial answer data; Determine whether the initial answer data meets the preset answer refinement requirements through the industry large model; the answer refinement requirements include that the target question type belongs to a specified question type, and the data volume of the initial answer data is greater than the preset data volume threshold; If the answer refinement requirements are met, extract the key steps from the initial answer data through the industry large model to obtain the question-and-answer result data.

7. The method according to claim 1, characterized in that, The voice conversion of the question-and-answer result data to obtain the response voice data includes: Using regular formatting and natural language processing techniques, convert the data format of the question-and-answer result data to obtain target text data, which conforms to the preset language habits and the production site voice broadcast requirements of the industrial and mining production environment; Use the preset voice synthesis model to perform voice synthesis on the target text data to obtain the response voice data.

8. A data question-and-answer device based on an intercom terminal, characterized in that, Applied to the large model business system, the large model business system is communicatively connected to the intercom terminal; the device includes: A first conversion module, configured to, when the request voice data sent by the intercom terminal in the industrial and mining production environment is obtained, perform text conversion on the request voice data to obtain the text information to be queried; A type recognition module, configured to use the trained industry classification model to identify the question type of the text information to be queried to obtain the target question type; wherein, the industry classification model is trained based on the industry knowledge corresponding to the industrial and mining production environment; A data query module, configured to, according to the target question type, use the trained industry large model to query the text information to be queried to obtain the question-and-answer result data; wherein, the industry large model is trained based on the industry knowledge question-and-answer samples; A second conversion module, configured to perform voice conversion on the question-and-answer result data to obtain the response voice data; A data sending module, configured to send the response voice data to the intercom terminal so that the intercom terminal performs voice broadcast on the response voice data.

9. A large model business system, including a memory and a processor, where a computer program that can run on the processor is stored in the memory, and is characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1-7.

10. A data question-and-answer system based on an intercom terminal, characterized in that, Including an intercom terminal and the large model business system according to claim 9, the large model business system is communicatively connected to the intercom terminal.

Images