Work flow compliance examination method and device based on voice recognition

Abstract

The invention provides a work flow compliance examination method and device based on voice recognition, and the method comprises the steps: carrying out the recognition and conversion of voice data collected in a target work scene, and obtaining target text data; inputting the target text data into a pre-trained language model, and outputting a plurality of groups of entities, events, association relationships between entities and entities and / or between events and between events; according to the plurality of groups of entities, events and association relationships, obtaining a to-be-reviewed operation process by using a target operation process template corresponding to the target operation scene; and performing compliance examination on the to-be-examined operation process to obtain an examination result. According to the scheme, automatic, intelligent and high-adaptability work flow compliance examination of voice dialogue in the whole process of work is achieved, the examination coverage rate is high, and the method is suitable for complex and changeable scenes.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a method and device for operation process compliance review based on speech recognition. Background Art

[0002] In the operation scenarios of many industries such as logistics, mining, railways, aviation, and industry, the whole process of communication between on-site operators and dispatching commanders mainly relies on voice communication methods such as walkie-talkies, radios, and mobile phones. In order to ensure the safety and quality of operations, many actions need to be subject to a set of standardized inspection processes before implementation, so these voice conversations need to comply with relevant operation specifications and requirements.

[0003] At present, compliance checks on operational voice dialogues mainly include manual sampling, keyword matching, and rule engine methods. Among them, the manual sampling method manually extracts some voice recordings for manual listening and evaluation, but it relies on manual operation, has a large workload and low efficiency, and it is difficult to achieve comprehensive coverage; the keyword matching method converts voice into text, and then matches it through a preset keyword vocabulary to check whether there are any illegal terms. Although this method can achieve automated review, it has limited semantic understanding capabilities in complex scenarios and is prone to misjudgment; the rule engine method uses preset business rules and logic to analyze and conduct compliance review on the text converted from voice. Although this method can handle certain complex logic, the formulation and maintenance of rules are complex and lack flexibility. It is difficult to handle violations beyond the preset range and difficult to cope with changing scenarios. Therefore, a method and device for compliance review of operational processes based on voice recognition is needed. Summary of the invention

[0004] The present invention provides a method and device for operation process compliance review based on voice recognition, which realizes automated, intelligent and adaptable operation process compliance review of voice dialogue throughout the entire operation process, and has high review coverage and is suitable for complex and changeable scenarios.

[0005] In a first aspect, the present invention provides a method for compliance review of an operation process based on speech recognition, comprising:

[0006] Recognize and convert the voice data collected in the target operation scene to obtain the target text data;

[0007] Inputting the target text data into a pre-trained language model, and outputting a plurality of groups of entities, events, association relationships between entities and entities and / or events, and between events and events;

[0008] According to the plurality of groups of entities, events and associations, a target operation process template corresponding to the target operation scenario is used to obtain the operation process to be reviewed;

[0009] Conduct a compliance review on the work process to be reviewed and obtain a review result.

[0010] Optionally, the recognition and conversion of the voice data collected in the target operation scene to obtain the target text data includes:

[0011] Preprocessing the voice data to obtain preprocessed voice data;

[0012] Marking the preprocessed voice data with timestamp and location information;

[0013] Performing recognition conversion on the preprocessed voice data to obtain temporary text data;

[0014] Correcting and semantically correcting the temporary text data according to the dialect knowledge base to obtain initial text data;

[0015] The initial text data is sorted according to the time sequence of the timestamps, and the position information is correspondingly added to the initial text data to obtain the target text data.

[0016] Optionally, the pre-trained language model is trained by the following method:

[0017] Acquire and freeze the training parameters of the language model; wherein the language model is used to identify the full-process text data of the operation in the historical text data in the target operation scene; the full-process text data of the operation includes several groups of entities, events, and association relationships;

[0018] Constructing a low-rank matrix in the architecture of the pre-trained language model to obtain a fine-tuned language model;

[0019] At least two groups of first sample sets are used to train the fine-tuning language model to obtain the pre-trained language model; wherein each group of the first sample sets includes historical text data in the target job scenario as input and several groups of entities, events, and association relationships of the historical text data as output.

[0020] Optionally, the language model is trained by the following method:

[0021] Analyze the historical text data in the target operation scene to obtain the full-process text data of the operation in the target operation scene;

[0022] The language model is obtained by training at least two groups of second sample sets; wherein each group of the second sample sets includes the historical text data as input and the full-process text data of the operation of the historical text data as output.

[0023] Optionally, the entities include the department to which the staff member belongs, the staff member, the equipment name, the work area, the work location, and the working hours; the events include the action and current status of performing any work task; the actions include completing the inspection, waiting for commands, starting construction, and completing the construction; the current status includes: the staff are all present, the equipment is being inspected, the equipment is ready, and the equipment has completed the work; the association relationships include the superior-subordinate relationships between the work departments to which the staff member belongs, and the sequence of the work tasks.

[0024] Optionally, obtaining the operation process to be reviewed based on the plurality of groups of entities, events and associations using a target operation process template corresponding to the target operation scenario includes:

[0025] Acquire the target operation process template adopted by the target operation scenario; wherein different operation process templates are adopted by different operation scenarios; the target operation process template is used to output the target entity, target event and target association relationship according to the specified output format;

[0026] According to the specified output format, determining a plurality of groups of the target entities, the target events, and the target association relationships from the plurality of groups of the entities, the events, and the association relationships;

[0027] Outputting a plurality of groups of the target entities, the target events and the target association relationships according to the specified output format and time sequence to obtain the job flow to be reviewed.

[0028] Optionally, the performing of a compliance review on the operation process to be reviewed to obtain a review result includes:

[0029] Obtaining a standard operating procedure for the target operating scenario;

[0030] Compare the standard operating procedures with the operating procedures to be reviewed to determine the review results including non-compliant process nodes and potential risk process nodes.

[0031] Optionally, comparing the standard operating procedure with the operating procedure to be reviewed to determine the review results including non-compliant process nodes and potential risk process nodes includes:

[0032] Segmenting the operation flow to be reviewed to obtain a first type of operation flow corresponding to the standard operation flow and a second type of operation flow not corresponding to the standard operation flow;

[0033] Review the first type of operating procedures according to the standard operating procedures to determine non-compliant process nodes;

[0034] Identify each of the second-category operation processes as a potential risk process node;

[0035] Perform risk assessment on the non-compliant process nodes and the potential risk process nodes to obtain risk scores, and determine solutions based on the risk scores.

[0036] In a second aspect, the present invention provides a work process compliance review device based on voice recognition, comprising:

[0037] The speech recognition module is used to recognize and convert the speech data collected in the target operation scene to obtain the target text data;

[0038] An analysis module is used to input the target text data into a pre-trained language model, output a plurality of groups of entities, events, associations between entities and entities and / or events, and between events and events; and obtain the work process to be reviewed using a target work process template corresponding to the target work scenario based on the plurality of groups of entities, events, and associations;

[0039] The review module is used to conduct a compliance review on the work process to be reviewed and obtain a review result.

[0040] In a third aspect, an embodiment of the present invention further provides a computing device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any first aspect of this specification is implemented.

[0041] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method described in any first aspect of this specification.

[0042] In a fifth aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method described in any first aspect of this specification are implemented.

[0043] The embodiment of the present invention provides a method and device for compliance review of operation processes based on speech recognition, which obtains target text data by recognizing and converting the speech data collected in the target operation scene, and then inputs the target text data into a pre-trained language model to output key information in the target operation scene, including entities, events, relationships between entities and entities and / or events, and between events and events, etc., and then uses the target operation process template under the target operation scene to output the operation process to be reviewed, and finally obtains the review result by conducting a compliance review on the operation process to be reviewed. In this way, this scheme uses a pre-trained language model with strong semantic understanding capabilities, without the need to design extraction rules, and can determine entities, events and their relationships based solely on concepts, attributes, etc. in the target text data. At the same time, through a specific target operation process template, the operation process information is accurately extracted, thereby breaking through the limitations of traditional keyword matching, achieving a deep understanding of complex operation scenes, and thus improving the accuracy and flexibility of compliance review. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 It is a flow chart of a method for compliance review of an operation process based on speech recognition provided by an embodiment of the present invention;

[0046] Figure 2 is a hardware architecture diagram of a computing device provided by an embodiment of the present invention;

[0047] Figure 3 It is a structural diagram of a work process compliance review device based on voice recognition provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0049] The specific implementation of the concept of this application is described below.

[0050] Please refer to Figure 1 The embodiment of the present invention provides a method for compliance review of an operation process based on speech recognition, the method comprising:

[0051] Step 100, recognizing and converting the voice data collected in the target operation scene to obtain target text data;

[0052] Step 102, inputting the target text data into a pre-trained language model, and outputting a plurality of groups of entities, events, associations between entities and entities and / or events, and between events and events;

[0053] Step 104, according to a plurality of groups of entities, events and associations, using a target operation process template corresponding to the target operation scenario to obtain the operation process to be reviewed;

[0054] Step 106, conduct a compliance review on the work process to be reviewed and obtain the review result.

[0055] In an embodiment of the present invention, the target text data is obtained by recognizing and converting the voice data collected in the target operation scene, and then the target text data is input into a pre-trained language model to output the key information in the target operation scene, including entities, events, relationships between entities and entities and / or events, and relationships between events, etc., and then the target operation process template under the target operation scene is used to output the operation process to be reviewed, and finally the review result is obtained by conducting a compliance review on the operation process to be reviewed. In this way, this solution uses a pre-trained language model with strong semantic understanding capabilities, without the need to design extraction rules. It can determine entities, events and their relationships based solely on concepts, attributes, etc. in the target text data. At the same time, through a specific target operation process template, the operation process information is accurately extracted, thereby breaking through the limitations of traditional keyword matching, achieving a deep understanding of complex operation scenes, and thereby improving the accuracy and flexibility of compliance reviews.

[0056] Described below Figure 1 How the various steps are performed.

[0057] First, in step 100, the voice data collected in the target operation scene is recognized and converted to obtain target text data, including:

[0058] Preprocessing the voice data to obtain preprocessed voice data;

[0059] Marking the preprocessed voice data with timestamp and location information;

[0060] Recognize and convert the preprocessed voice data to obtain temporary text data;

[0061] Correct and semantically correct the temporary text data according to the dialect knowledge base to obtain the initial text data;

[0062] Sort the initial text data in chronological order of timestamps, and add position information to the corresponding positions in the initial text data to obtain the target text data.

[0063] Specifically, the present invention preferably uses a high-sensitivity microphone array or an intelligent intercom to capture the voice communication content, identity information, and position information among operators in real time in the target operation scenario. Among them, the position information is used to represent the geographical location of the current voice data in the target operation scenario. The preprocessing is used to improve the voice quality, including noise reduction, decibel enhancement, etc.

[0064] In the present invention, by marking the timestamps and position information for the voice data, the subsequent analysis efficiency and the operation process to be reviewed are improved. At the same time, since there are dialects in the actual operation scenario, it is also necessary to further use the dialect knowledge base to correct and semantically correct the temporary text data to improve the conversion accuracy from voice to text.

[0065] In a specific embodiment, step 100 can use a speech recognition model trained by a deep learning algorithm (such as a Transformer or Conformer architecture) and a dialect knowledge base to perform recognition and conversion on the voice data, specifically including: (1) constructing a dialect knowledge base: providing background information corresponding to the dialect and Mandarin for the speech recognition model to assist the correction process; (2) training the speech recognition model with dialect voices and their corresponding dialect texts, Mandarin voices, and corresponding Mandarin texts to obtain a trained speech recognition model; (3) inputting the voice data into the trained speech recognition model to output the target text data. In this way, the trained speech recognition model can recognize and label the dialects, providing a direction for correction; at the same time, it can also perform context understanding and correction, and correct the dialect transcription errors and semantically correct them through the powerful language understanding ability of the model, combined with the context and the dialect knowledge base, thereby improving the conversion accuracy from voice data to text data.

[0066] For example, in the dialect knowledge base, the dialect is "Nong hao fa", and its corresponding Mandarin is "Ni hao ma"; the dialect is "Ge ge shi qing za nong", and its corresponding Mandarin is "Zhe ge shi qing zen me ban".

[0067] In step 102, the pre-trained language model is trained by the following method:

[0068] S1: Recognize and convert the historical voice data in the obtained target operation scenario to obtain historical text data; analyze the historical text data in the target operation scenario to obtain the full-process text data of the operation in the target operation scenario;

[0069] S2: Extract key information from the text data of the entire operation process to obtain several groups of entities, events, and relationships in the target operation scene;

[0070] S3: The language model is trained by at least two sets of second sample sets; wherein each set of second sample sets includes historical text data as input and full-process text data of the historical text data as output

[0071] S4: Obtaining and freezing training parameters of a language model; wherein the language model is used to identify the full-process text data of the job in the historical text data in the target job scenario; the full-process text data of the job includes entities, events, and association relationships;

[0072] S5: construct a low-rank matrix in the architecture of the pre-trained language model to obtain a fine-tuned language model;

[0073] S6: Use at least two groups of first sample sets to train and fine-tune the language model to obtain a pre-trained language model; wherein each group of first sample sets includes historical text data in the target work scenario as input and several groups of entities, events, and association relationships of the historical text data as output.

[0074] It should be noted that each group of entities, events, and associations corresponds to a time node or time period, that is, the voice data at this time node or time period can be converted into a group of entities, events, and associations. Describing voice data through an entity-event-association group is conducive to improving the efficiency and accuracy of obtaining the work process.

[0075] Specifically, in step S1, the method of step 100 is used to perform recognition conversion to obtain historical text data, and the text data of the entire operation process is obtained by removing the text data in the historical text data that is not related to the operation process. In step S2, key information extraction adopts sentence component analysis to determine the subject, predicate, object, complement, attributive, adverbial, and appositive from the text data, and further determine the entity, event, and association relationship according to the determined sentence components. Entities are identified and determined from sentence components according to predefined probabilities, such as people, equipment, places, etc. The language model in step S3 is a large language model, such as GPT-3, LLaMA, etc.

[0076] Specifically, in steps S5 and S6, the low-rank matrix includes a first low-rank matrix and a second low-rank matrix, and for each layer of the language model architecture, the following steps are performed:

[0077] A first low-rank matrix and a second low-rank matrix are introduced into the layer, and an output of the layer is determined according to an input of the layer, training parameters of the language model, the first low-rank matrix and the second low-rank matrix;

[0078] Using at least two groups of first sample sets to train the fine-tuned language model after the low-rank matrix is ​​introduced, determining the first low-rank matrix and the second low-rank matrix corresponding to each layer, and obtaining a pre-trained language model;

[0079] The output of this layer is determined by the following formula: h = (W0 + BA) x

[0080] Among them, h is the output of this layer, x is the input of this layer, W0 is the training parameter of the language model, A is the first low-rank matrix, and B is the second low-rank matrix. The matrix dimensions of A and B are smaller than the matrix dimension of W0.

[0081] Specifically, when training the fine-tuning language model, the original training parameter W0 is frozen, and low-rank approximation training is performed using matrices A and B with smaller parameters. At the beginning of training, A is initialized with a random Gaussian, and B is initialized to an all-zero matrix to ensure that the training starts from W0. In this way, only A is trained by reducing the dimension and B is trained by increasing the dimension during training; after training, only the B and A parameters of the low-rank matrix need to be saved. The present invention greatly reduces the training overhead by constructing a low-rank matrix in the language model, and can quickly respond to updates when the fine-tuning language model is updated.

[0082] In a preferred embodiment, entities include the department to which the staff member belongs, the staff member, the equipment name, the work area, the work location, and the working hours; events include the action and current status of performing any work task; actions include completing inspection, waiting for orders, starting construction, and completing construction; current status includes: staff are present, equipment is being inspected, equipment is ready, and equipment has completed the work; association relationships include the superior-subordinate relationship between the work departments to which the staff member belongs, and the sequence of work tasks.

[0083] In the present invention, since the text data contains rich construction operation scene information, and involves multi-role participation, various possible events and instructions, in order to collect the operation process in the operation scene, it is necessary to extract key information from the operation process text data according to the standard operation process, operating specifications and safety requirements documents, so as to facilitate the determination of the operation process from the text data. Specifically, for example, in the electrical construction operation scene, the entity is the staff: such as "construction team", the department to which the staff belongs: such as "command center", "electrical inspection team", the equipment name: such as "equipment 031", "equipment 011", "equipment 052", the geographical location of the work: such as "construction area", "electrical equipment inspection area", and the working time: standardized time format. The event is an action: such as "complete inspection", "wait for order", "issue order", "start construction", "construction completed", and the current state: such as "staff are all present", "electrical equipment is being inspected", "electrical equipment is ready", and "electrical equipment completes the operation". The association relationship is a superior-subordinate relationship: for example, the "command center" issues an order to the "construction team". It is a sequence: for example, the electrical equipment inspection task is before the electrical equipment construction task, that is, the sequence of events. It is an execution relationship between entities and events: for example, the electrical inspection team conducts electrical equipment inspections.

[0084] In the present invention, by constructing entities, events and association relationship groups, it is possible to accurately extract key information in the conversation, automatically extract key information in the voice data without being restricted by the keyword library, adapt to different work scenarios, achieve coverage of the entire work process, avoid omissions in manual sampling, and flexibly determine a more accurate work process.

[0085] In step 104, according to a plurality of groups of entities, events and associations, the target operation process template corresponding to the target operation scenario is used to obtain the operation process to be reviewed, including:

[0086] Obtain a target operation process template used by a target operation scenario; wherein different operation process templates are used in different operation scenarios; the target operation process template is used to output a target entity, a target event, and a target association relationship according to a specified output format;

[0087] According to a specified output format, a plurality of groups of target entities, target events and target relationships are determined from a plurality of groups of entities, events and relationships;

[0088] Output several groups of target entities, target events and target relationships in a specified output format and time sequence to obtain the work flow to be reviewed.

[0089] For example, the specified output format of the job flow template is:

[0090] Event List:

[0091] {

[0092]

[0093]

[0094] In the present invention, the construction operation process is reconstructed using a work flow template with a specified output format to obtain the actual work flow (i.e., the work flow to be reviewed), making the work flow more visual and convenient for reviewing compliance; at the same time, different work flow templates are used for different work scenarios, so that the work flow under different work scenarios can be flexibly obtained.

[0095] In step 106, a compliance review is conducted on the work process to be reviewed, and the review results are obtained, including:

[0096] Obtain standard operating procedures for target operating scenarios;

[0097] Compare the standard operating procedures with the operating procedures to be reviewed to determine the review results, including non-compliant process nodes and potential risk process nodes.

[0098] Specifically, the work process to be reviewed is compared with the standard work process, and the deviations and non-compliance of the process steps are identified to obtain a list of differences. Among them, the compliance comparison includes: execution order comparison, judging whether the execution order of the work process to be reviewed is consistent with the execution order of the standard work process, and checking whether there are omissions of certain standard steps in the actual work process to be reviewed, and whether there are steps outside the standard process, and identifying the steps with inconsistent execution order and omission of a certain step as non-compliant process nodes; and identifying the steps outside the standard process as potential risk process nodes.

[0099] In a preferred embodiment, the standard operating process is compared with the operating process to be reviewed, and the review results including non-compliant process nodes and potential risk process nodes are determined, including:

[0100] The work flow to be reviewed is segmented to obtain the first type of work flow corresponding to the standard work flow and the second type of work flow not corresponding to the standard work flow;

[0101] Review the first category of operating procedures according to standard operating procedures and identify non-compliant process nodes;

[0102] Identify each Category II operational process as a potential risk process node;

[0103] Conduct risk assessments on non-compliant process nodes and potential risk process nodes, obtain risk scores, and determine solutions based on the risk scores.

[0104] For example, following the previous example, the step numbering sequence of the work process to be reviewed is 2, 1, 1, 3, 4, and the step numbering sequence of the standard work process is 1, 2, 3, 4. Therefore, step 2 of the work process to be reviewed is a non-compliant process node, and there is no potential risk process node. At this time, in the actual process, the construction team entered the construction area before the electrical equipment inspection was completed, which violated the safety production regulations and posed a safety hazard, which may lead to electrical accidents during the construction process; the reason is that the construction team did not strictly abide by the process and lacked confirmation of the electrical inspection status; the solution is to strengthen process training, and the construction team can only apply to enter the construction area after the electrical inspection is completed and approved by the command center. It should be noted that the step number outside the standard process can be "0".

[0105] In the present invention, since not all operating processes in actual operating processes correspond to standard operating processes, the second type of operating processes other than standard processes are all determined as potential risk process nodes, and by performing risk assessments on non-compliant process nodes and potential risk process nodes, the risk score of the current operating process to be reviewed can be further determined, thereby further determining a suitable solution.

[0106] In a specific embodiment, the risk score is determined by the following formula:

[0107]

[0108] Where M is the risk score; n1 is the number of non-compliant process nodes; α1, α2, and α3 are the weight values ​​corresponding to the number of operators detection results, precondition satisfaction results, and compliance results respectively; p i 、p ei are the actual number of people and the designated number of people working at the i-th non-compliant process node; m 2i The result of satisfying the precondition of the i-th non-compliant process node; m 3i is the compliance result of the i-th non-compliant process node; α1+α2+α3=1; n2 is the number of potential risk process nodes; m 4j is the risk score of the jth potential risk process node; β is the impact factor of the pending operation process on the current operation, β>0. The higher the risk score, the greater the risk.

[0109] It should be noted that m 2i 、m 3i The value of is 0 or 1, 0<m 4j ≤1. Among them, m 2i =0, to meet the precondition; m 2i =1, the precondition is not met. 3i=0, the compliance result meets the specification requirements; m 3i =1, the compliance result does not meet the specification requirements. 4j Determined based on experience. β is related to the impact of the pending job flow on the current job operation. The greater the impact on the current job operation, the higher the β value. If the current job can run stably under the pending job flow, the β value is the smallest.

[0110] In this specific implementation, the risk assessment of the non-compliant process node includes whether the preconditions of the step where the non-compliant process node is located are met, whether the execution of the step where the non-compliant process node is located meets the specification requirements, and whether the number of operators meets the specified number of operators. For non-compliant process nodes, if all three results are not met, the higher the risk, the higher the security risk. It should be noted that the precondition is used to determine whether the preconditions for the execution of the current non-compliant process node are met. Only when the preconditions are met, the current non-compliant process node is more compliant.

[0111] In the present invention, the higher the risk score, the higher the risk level, the higher and more urgent the potential safety hazards in the current operating process, and the more urgently the operation and maintenance personnel need to adopt corresponding solutions to perform safety inspections and adjustments.

[0112] In the present invention, by automatically collecting and processing all voice data in the operation process, comprehensive coverage of the entire operation process is achieved, omissions caused by manual sampling are avoided, and the review coverage rate is improved. The powerful semantic understanding ability of the large language model is used to accurately extract the operation process information in the text data, avoid misjudgment caused by simple keyword matching, and improve the accuracy of compliance review. At the same time, the pre-trained language model is used for intelligent analysis, which reduces the dependence on complex rule engines, reduces the complexity and cost of rule maintenance, and simplifies rule maintenance. Moreover, through the continuous learning and updating of the large language model, the operation process compliance review method provided by the present invention can flexibly respond to changes in operation specifications in different industries and different scenarios, quickly adapt to new compliance requirements, and has higher adaptability. In addition, compliance review can also determine violations, identify potential risks and hidden dangers, and provide more comprehensive protection for operation safety and quality; and when the review results are non-compliant, a clear explanation and solution are given to facilitate relevant personnel to understand and improve. The present invention significantly improves the compliance review efficiency and accuracy of operation voice dialogues, and provides strong support for operation safety and quality management in logistics, mining, railways, aviation and other industries.

[0113] like Figure 2 , Figure 3 As shown, an embodiment of the present invention provides a device for compliance review of operation flow based on speech recognition. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Figure 2 As shown in FIG. 1 , a hardware architecture diagram of a computing device in which a voice recognition-based operation process compliance review device is provided in an embodiment of the present invention is provided. Figure 2 In addition to the processor, memory, network interface, and non-volatile memory shown in the figure, the computing device in which the device is located in the embodiment may also generally include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 3 As shown, as a device in a logical sense, the CPU of the computing device in which it is located reads the corresponding computer program in the non-volatile memory into the memory and runs it. This embodiment provides a work process compliance review device based on voice recognition, which includes:

[0114] The speech recognition module 300 is used to recognize and convert the speech data collected in the target operation scene to obtain target text data;

[0115] The analysis module 302 is used to input the target text data into the pre-trained language model, output a plurality of groups of entities, events, associations between entities and entities and / or events, and between events and events; and obtain the work process to be reviewed using the target work process template corresponding to the target work scenario according to the plurality of groups of entities, events and associations;

[0116] The review module 304 is used to perform a compliance review on the work process to be reviewed and obtain a review result.

[0117] In some specific implementations, the speech recognition module 300 may be used to execute the above step 100 , the analysis module 302 may be used to execute the above steps 102 and 104 , and the review module 304 may be used to execute the above step 106 .

[0118] In one embodiment of the present invention, entities include the department to which the staff member belongs, the staff member, the equipment name, the work area, the work location, and the working hours; events include the action and current status of performing any work task; actions include completing inspection, waiting for commands, starting construction, and completing construction; current status includes: staff members are present, equipment is being inspected, equipment is ready, and equipment has completed the work; association relationships include the superior-subordinate relationship between the work departments to which the staff members belong, and the sequence of work tasks.

[0119] In one embodiment of the present invention, the speech recognition module 300 is further configured to perform the following operations:

[0120] Preprocessing the voice data to obtain preprocessed voice data;

[0121] Marking the preprocessed voice data with timestamp and location information;

[0122] Recognize and convert the preprocessed voice data to obtain temporary text data;

[0123] Correcting and semantically correcting temporary text data according to the dialect knowledge base to obtain initial text data;

[0124] The initial text data is sorted in the chronological order of the timestamps, and the position information is added to the initial text data accordingly to obtain the target text data.

[0125] In one embodiment of the present invention, a training module is further included, and the training module is used to perform the following operations:

[0126] The training parameters of the language model are obtained and frozen; wherein the language model is used to identify the full-process text data of the operation in the historical text data in the target operation scene; the full-process text data of the operation includes several groups of entities, events, and association relationships;

[0127] Construct a low-rank matrix in the architecture of the pre-trained language model to obtain a fine-tuned language model;

[0128] At least two groups of first sample sets are used to train and fine-tune the language model to obtain a pre-trained language model; wherein each group of first sample sets includes historical text data in a target job scenario as input and several groups of entities, events, and association relationships of the historical text data as output.

[0129] In one embodiment of the present invention, the training module is further used to perform the following operations:

[0130] Analyze the historical text data in the target operation scene to obtain the full-process text data of the operation in the target operation scene;

[0131] The language model is trained by at least two groups of second sample sets; wherein each group of second sample sets includes historical text data as input and full-process text data of the historical text data as output.

[0132] In one embodiment of the present invention, the analysis module 302 is further configured to perform the following operations:

[0133] Obtain a target operation process template used by a target operation scenario; wherein different operation process templates are used in different operation scenarios; the target operation process template is used to output a target entity, a target event, and a target association relationship according to a specified output format;

[0134] According to a specified output format, a plurality of groups of target entities, target events and target relationships are determined from a plurality of groups of entities, events and relationships;

[0135] Output several groups of target entities, target events and target relationships in a specified output format and time sequence to obtain the work flow to be reviewed.

[0136] In one embodiment of the present invention, the review module 304 is further configured to perform the following operations:

[0137] Obtain standard operating procedures for target operating scenarios;

[0138] The work flow to be reviewed is segmented to obtain the first type of work flow corresponding to the standard work flow and the second type of work flow not corresponding to the standard work flow;

[0139] Review the first category of operating procedures according to standard operating procedures and identify non-compliant process nodes;

[0140] Identify each Category II operational process as a potential risk process node;

[0141] Conduct risk assessments on non-compliant process nodes and potential risk process nodes, obtain risk scores, and determine solutions based on the risk scores.

[0142] It is understandable that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on a device for compliance review of a work process based on voice recognition. In other embodiments of the present invention, a device for compliance review of a work process based on voice recognition may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0143] The information interaction, execution process and other contents between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For specific contents, please refer to the description in the embodiment of the method of the present invention, and no further description is given here.

[0144] An embodiment of the present invention further provides a computing device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a method for compliance review of an operation process based on speech recognition in any embodiment of the present invention is implemented.

[0145] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes a method for compliance review of a job process based on speech recognition in any embodiment of the present invention.

[0146] An embodiment of the present application also provides a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium, and the processor executes the computer program, so that the computer device executes a job process compliance review method based on voice recognition as described in any of the above embodiments.

[0147] Specifically, a system or device equipped with a storage medium can be provided, on which software program code that implements the functions of any of the above-mentioned embodiments is stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program code stored in the storage medium.

[0148] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0149] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0150] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling the operating system on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0151] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0152] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical factors in the process, method, article or device including the elements.

[0153] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for compliance review of operation processes based on speech recognition, characterized in that: include: Recognize and convert the voice data collected in the target operation scene to obtain the target text data; Inputting the target text data into a pre-trained language model, and outputting a plurality of groups of entities, events, association relationships between entities and entities and / or events, and between events and events; According to the plurality of groups of entities, events and associations, a target operation process template corresponding to the target operation scenario is used to obtain the operation process to be reviewed; Conduct a compliance review on the work process to be reviewed and obtain a review result.

2. The method according to claim 1, characterized in that The method of recognizing and converting the voice data collected in the target operation scene to obtain target text data includes: Preprocessing the voice data to obtain preprocessed voice data; Marking the preprocessed voice data with timestamp and location information; Performing recognition conversion on the preprocessed voice data to obtain temporary text data; Correcting and semantically correcting the temporary text data according to the dialect knowledge base to obtain initial text data; The initial text data is sorted according to the time sequence of the timestamps, and the position information is correspondingly added to the initial text data to obtain the target text data.

3. The method according to claim 1, characterized in that The pre-trained language model is trained by the following method: Acquire and freeze the training parameters of the language model; wherein the language model is used to identify the full-process text data of the operation in the historical text data in the target operation scene; the full-process text data of the operation includes several groups of entities, events, and association relationships; Constructing a low-rank matrix in the architecture of the pre-trained language model to obtain a fine-tuned language model; At least two groups of first sample sets are used to train the fine-tuning language model to obtain the pre-trained language model; wherein each group of the first sample sets includes historical text data in the target job scenario as input and several groups of entities, events, and association relationships of the historical text data as output.

4. The method according to claim 3, characterized in that The language model is trained by the following method: Analyze the historical text data in the target operation scene to obtain the full-process text data of the operation in the target operation scene; The language model is obtained by training at least two groups of second sample sets; wherein each group of the second sample sets includes the historical text data as input and the full-process text data of the operation of the historical text data as output.

5. The method according to claim 1, characterized in that The entities include the department to which the staff member belongs, the staff member, the equipment name, the work area, the work geographical location, and the work time; the events include the action and current status of executing any work task; the actions include completing the inspection, waiting for the command, starting the construction, and completing the construction; the current status includes: the staff are all present, the equipment is being inspected, the equipment is ready, and the equipment has completed the work; the association relationships include the superior-subordinate relationship between the work departments to which the staff member belongs, and the sequence of the work tasks; and / or, The step of obtaining the operation process to be reviewed by using the target operation process template corresponding to the target operation scenario according to the plurality of groups of the entities, the events and the association relationships includes: Acquire the target operation process template adopted by the target operation scenario; wherein different operation process templates are adopted by different operation scenarios; the target operation process template is used to output the target entity, target event and target association relationship according to the specified output format; According to the specified output format, determining a plurality of groups of the target entities, the target events, and the target association relationships from the plurality of groups of the entities, the events, and the association relationships; Outputting a plurality of groups of the target entities, the target events and the target association relationships according to the specified output format and time sequence to obtain the job flow to be reviewed.

6. The method according to any one of claims 1 to 5, characterized in that: The compliance review of the operation process to be reviewed is performed to obtain the review results, including: Obtaining a standard operating procedure for the target operating scenario; Compare the standard operating procedures with the operating procedures to be reviewed to determine the review results including non-compliant process nodes and potential risk process nodes.

7. The method according to claim 6, characterized in that The comparison between the standard operating process and the operating process to be reviewed to determine the review results including non-compliant process nodes and potential risk process nodes includes: Segmenting the operation flow to be reviewed to obtain a first type of operation flow corresponding to the standard operation flow and a second type of operation flow not corresponding to the standard operation flow; Review the first type of operating procedures according to the standard operating procedures to determine non-compliant process nodes; Identify each of the second-category operation processes as a potential risk process node; Perform risk assessment on the non-compliant process nodes and the potential risk process nodes to obtain risk scores, and determine solutions based on the risk scores.

8. A device for reviewing compliance of operation flow based on speech recognition, characterized in that: include: The speech recognition module is used to recognize and convert the speech data collected in the target operation scene to obtain the target text data; An analysis module is used to input the target text data into a pre-trained language model, output a plurality of groups of entities, events, associations between entities and entities and / or events, and between events and events; and obtain the work process to be reviewed using a target work process template corresponding to the target work scenario based on the plurality of groups of entities, events, and associations; The review module is used to conduct a compliance review on the work process to be reviewed and obtain a review result.

9. A computing device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 7.

Images